-- Logs begin at Thu 2022-10-06 07:21:42 UTC, end at Thu 2022-10-06 08:40:07 UTC. -- Oct 06 07:52:26.022687 dv-cinder-860212-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 06 07:52:26.032540 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 06 07:52:26.034737 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Oct 6 07:52:26 2022] *** Oct 06 07:52:26.034737 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 06 07:52:26.034737 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 06 07:52:26.034737 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: nodename: dv-cinder-860212-3-iscsi Oct 06 07:52:26.034737 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: machine: x86_64 Oct 06 07:52:26.034737 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: clock source: unix Oct 06 07:52:26.034737 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: pcre jit disabled Oct 06 07:52:26.035209 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: detected number of CPU cores: 4 Oct 06 07:52:26.035209 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: current working directory: / Oct 06 07:52:26.035209 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: detected binary path: /usr/bin/uwsgi-core Oct 06 07:52:26.035209 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: your processes number limit is 47680 Oct 06 07:52:26.035209 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: your memory page size is 4096 bytes Oct 06 07:52:26.035209 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: detected max file descriptor number: 2048 Oct 06 07:52:26.035209 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: lock engine: pthread robust mutexes Oct 06 07:52:26.035209 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: thunder lock: enabled Oct 06 07:52:26.035779 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 06 07:52:26.035927 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 06 07:52:26.076976 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: Python main interpreter initialized at 0x555cb43049c0 Oct 06 07:52:26.076976 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: python threads support enabled Oct 06 07:52:26.076976 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: your server socket listen backlog is limited to 100 connections Oct 06 07:52:26.076976 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: your mercy for graceful operations on workers is 90 seconds Oct 06 07:52:26.077362 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: mapped 403077 bytes (393 KB) for 2 cores Oct 06 07:52:26.077441 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: *** Operational MODE: preforking *** Oct 06 07:52:26.077691 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: *** uWSGI is running in multiple interpreter mode *** Oct 06 07:52:26.077691 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: spawned uWSGI master process (pid: 81467) Oct 06 07:52:26.077907 dv-cinder-860212-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 06 07:52:26.080352 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: spawned uWSGI worker 1 (pid: 81479, cores: 1) Oct 06 07:52:26.080352 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: spawned uWSGI worker 2 (pid: 81480, cores: 1) Oct 06 07:52:26.080352 dv-cinder-860212-3-iscsi devstack@keystone.service[81467]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 06 07:52:30.439461 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 06 07:52:30.439461 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: from cryptography.utils import int_from_bytes Oct 06 07:52:30.440204 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 06 07:52:30.440204 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: from cryptography.utils import int_from_bytes Oct 06 07:52:30.440611 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 06 07:52:30.440611 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: from cryptography.utils import int_from_bytes Oct 06 07:52:30.441057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 06 07:52:30.441057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: from cryptography.utils import int_from_bytes Oct 06 07:52:30.493189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 06 07:52:30.493384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 06 07:52:30.493557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] command line args: [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 06 07:52:30.493874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 06 07:52:30.493967 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 06 07:52:30.494050 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ================================================================================ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 06 07:52:30.494134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 06 07:52:30.494211 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] command line args: [] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 06 07:52:30.494281 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] admin_token = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.494465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 06 07:52:30.494549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] config_dir = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.494646 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ================================================================================ {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 06 07:52:30.494732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.494830 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] config_source = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.494907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] admin_token = **** {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.495046 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] debug = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.495194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] config_dir = [] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.499437 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.499437 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] config_source = [] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.499570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.499790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] debug = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.499898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.500004 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.500129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.500232 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.500333 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.500444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.500539 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.500639 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.500740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.500844 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.500942 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_config_append = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.501060 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] list_limit = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.501139 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.501264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] log_config_append = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.501344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_dir = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.501460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.501538 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_file = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.501652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] log_dir = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.501730 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.501854 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] log_file = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.501928 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.502041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.502120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.502190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.502262 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.502342 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.502412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.502528 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.502603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.502716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.502844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.502919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.503052 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.503132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.503245 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.503323 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.503402 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.503472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.503540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.503652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.503724 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.503799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.503875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.503945 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.504030 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.504099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.504167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.504242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.504242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.504428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.504428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] publish_errors = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.504597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.504721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.504800 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] publish_errors = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.504875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.504993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.505065 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.505149 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.505220 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.505293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.505381 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.505462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.505537 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] standard_threads = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.505607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.505721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.505800 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] standard_threads = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.505880 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.505956 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.506030 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.506030 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_journal = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.506158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.506232 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_json = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.506312 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.506383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_stderr = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.506451 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] use_journal = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.506519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_syslog = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.506587 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] use_json = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.506660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.506741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] use_stderr = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.506816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.506886 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] use_syslog = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.506976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.507058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 06 07:52:30.507134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.507282 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.507362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.507437 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.507511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.507581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.507648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.507726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.507796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.507864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.507937 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.508010 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.508080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.508157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.508232 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.508439 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.508515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.508585 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.508660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.508741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.508811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.508879 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.508947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.509044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.509119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.509256 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.509331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.509406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.509475 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.509543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.509621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.509691 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.509749 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.509815 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.509883 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.509950 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.510025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.510100 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.510168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.510235 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.510303 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.510369 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.510447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.510577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.510658 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.511406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.511516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.511902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.512167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.512247 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.512409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.512486 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.512651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.512728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.512890 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.512967 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.513134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.513212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.513389 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.513466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.513637 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.513721 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.513874 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.513955 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.514110 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.514192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.514363 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.514440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.514616 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.514691 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.514856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.514933 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.515132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.515209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.515423 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.515544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.515667 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.515795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.515900 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.515998 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.516126 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.516235 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.516364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.516469 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.516565 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.516659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.516803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.516915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.517028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.517148 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.517276 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.517402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.517521 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.517634 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.517746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.517866 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.517988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.518103 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.518214 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.518325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.518437 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.518545 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.518657 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.518775 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.518889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.519049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.519156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.519267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.519386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.519498 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.519611 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.519730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.519833 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.519944 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.520067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.520179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.520291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.520409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.520518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.520635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.520760 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.520874 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.520994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.521099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.521217 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.521330 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.521451 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.521572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.521686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.521804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.521938 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.522054 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.522166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.522288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.522403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.522515 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.522628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.522746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.522859 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.522998 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.523126 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.523241 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.523353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.523466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.523577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.523768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.523890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.524009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.524128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.524241 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.524361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.524482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.524602 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.524715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.524832 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.524946 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.525067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.525189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.525310 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.525430 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.525544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.525663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.525777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.525890 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.526010 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.526124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.526237 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.526349 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.526463 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.526638 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.527330 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.527446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.527567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.527871 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.528002 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.528123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.528246 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.528366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.528487 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.528608 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.528722 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.528835 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.528960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.529074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.529188 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.529307 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.529421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.529533 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.529655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.529769 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.529882 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.530000 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.530115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.530228 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.530348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.530463 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.530576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.530689 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.530802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.530915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.531060 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.531176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.531290 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.531403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.531516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.531629 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.531752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.531866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.531982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.532096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.532251 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.532395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.external = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.532519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.532634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.532758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] auth.external = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.532874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.532995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.533109 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.533223 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.password = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.533336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.533591 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.token = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.533714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.533829 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.533952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] auth.password = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.534069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.534183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] auth.token = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.534305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.534421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.534525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.534637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.534751 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.534877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.535025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.535144 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.535259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.535373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.535487 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.535606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.535730 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.535855 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.535978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.536093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] credential.caching = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.536207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.536320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.536442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.536557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.536671 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.536784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.536898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.537020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.537141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.537255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.537369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.537482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.537596 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.537717 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.537832 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.537953 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.538077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.538192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.538305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.538494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.538612 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.538726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.538840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.539001 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.539119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.539245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.539361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.539475 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.539589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.539702 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.539816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.539938 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] federation.caching = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.540054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.540169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.540282 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.540395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.540509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.540631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.540780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.540898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.540996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.541091 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.541187 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.541289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.541386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.541482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.541577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.541672 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.541766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.541877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.542001 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.542144 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.542548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.542768 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.542872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity.caching = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.545231 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.545348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.545430 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.545502 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.545572 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.545642 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.545721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.545794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.545864 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.545934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.546004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.546073 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.546151 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.546222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.546293 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.546363 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.546433 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.546502 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.546580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.546651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.546722 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.546791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.546861 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.546931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.547040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.547114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.547190 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.547261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.547331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.547402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.547483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.547556 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.547626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.547695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.547765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.547844 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.547916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.547986 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.548055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.548129 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.548199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.548276 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.548347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.548417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.548491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.548562 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.548631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.548708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.548779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.548849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.548919 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.549043 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.549122 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.549299 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.549378 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.549538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.549615 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.549792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.549868 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.550041 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.550120 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.550283 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.550359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.550571 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.550653 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.550806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.550919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.551091 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.551235 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.551371 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.551490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.551621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.551749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.551875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.552001 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.552132 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.552257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.552257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.552518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.552646 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.552772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.552896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.553020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.553153 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.553279 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.553403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.553528 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.553652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.553779 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.553911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.554037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.554162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.554297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.554422 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.554546 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.554678 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.554807 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.554932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.555104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.555229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.555353 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.555488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.555614 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.555738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.555864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.555998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.556124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.556256 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.556381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.556506 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.556656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.556778 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.556902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.557035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.557161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.557286 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.557411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.557535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.557660 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.557797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.557919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.558045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.558170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.558295 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.558419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.558419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.558419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.558773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.558898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.559053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.559178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.559313 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.559439 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.559574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.559699 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.559824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.559949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.560083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.560206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.560330 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.560455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.560580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.560711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.560837 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.561218 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.561347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.561481 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.561608 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.561740 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.561866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.562014 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.562155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.562284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.562401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.562592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.562710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.562823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.562983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.563100 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.563216 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.563340 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.563456 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.563569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.563682 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.563795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.563907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.564032 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.564147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.564262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.564375 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.564488 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.564600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.564721 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.564836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.564949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] resource.caching = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.565061 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.565174 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.565288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.565409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.565522 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.565635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.565747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.565860 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.565972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.566092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.566207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.566321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.566433 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.566433 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.566644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.566644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.566870 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.567015 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.567131 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] role.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.567244 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] role.caching = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.567358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] role.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.567358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.567578 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] role.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.567578 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.567789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.567902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.568014 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.568140 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.568255 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.568367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.568480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.568480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.568730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.568837 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.568933 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.569026 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.569121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.569214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.569308 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.569411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.569508 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.569602 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.569728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.569831 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.569927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.570034 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.570131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.570227 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.570321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.570321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.570518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.570518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.570694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.570836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.570955 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.571056 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.573906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.574017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.574120 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.574200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.574271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.574348 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.574428 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.574501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.574571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.574641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.574711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.574781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.574859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.574931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.575024 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.575095 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.575165 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.575235 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.575317 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.575391 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.575461 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.575531 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.575600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.575600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.575600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.575792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.575862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.575932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.576001 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.576070 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.576146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.576218 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.576287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] token.caching = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.576356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.576425 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.576494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.576571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.576704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.576782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.576907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.576983 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.577054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.577132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.577204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.577274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.577343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.577413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.577483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.577560 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.577632 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.577702 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.577818 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.577892 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.577962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.578040 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.578112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.578236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.578314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.578489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.578566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.578766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.578845 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.579023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.579145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.579322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.579409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.579592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.579670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.579852 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.579931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.connection = **** {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.580098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.580176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.580351 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.580480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.580611 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.580735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.580857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.580978 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.581110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.581232 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.581428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.581551 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.581673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.581795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.581916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.582038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.582166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.582289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.582410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.582532 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.582654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.582775 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.582913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.583064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.583181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.583306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.583422 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.583561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.583686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.583802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.583916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.584029 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.584143 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.584257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.584377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.584492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 06 07:52:30.584605 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 06 07:52:30.584720 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81479) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 06 07:52:30.614887 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.615141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.615246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.615369 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.615452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.615647 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.624211 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.624666 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.656591 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.656822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.657481 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.657717 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.717404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.717937 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.768210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.768445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.768831 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.769067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.941573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.941801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.941979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.942067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.942215 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.942322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.942440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.942522 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.942662 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.942743 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.942880 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.943189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.966319 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.966554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.966781 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.967019 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.967112 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.967189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.967262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.967335 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.967481 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.967580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.967657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.967730 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.967922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.968135 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.978623 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:30.979104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:31.011847 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:31.012082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:31.012253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:31.012353 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:31.012496 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:31.012579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:31.012719 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:31.012799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:31.012933 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:31.013015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:31.013146 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:31.013227 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81479) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:31.013359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:31.013573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 06 07:52:31.030933 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.031159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 06 07:52:31.031511 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.031593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.031713 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 06 07:52:31.031810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 06 07:52:31.031994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.032107 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.032257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.032337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 06 07:52:31.032517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.032606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.032779 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.032858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.033066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.033326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.043182 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.043463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.043737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.044026 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.044106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.044349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.044625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.044910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.051239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.051903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.054227 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.054514 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.054799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.054879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.055109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.055193 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.055397 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.055482 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.055661 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.055743 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.055935 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.056017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.056202 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.056283 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.056473 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.056553 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.056745 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.056832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.057016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.057096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.057286 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.057555 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.079942 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.081147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.081405 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.081703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.082674 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.082978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.086284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.086572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.086856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.087658 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.087944 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.088230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.094112 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.094404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.095675 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.095969 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.098364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.098662 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.098966 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.099261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.101257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.101559 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.101652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.101853 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.101934 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.102153 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.102235 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.102451 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.102731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.103035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.103314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.103586 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.118417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.118701 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.120717 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.121013 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.122895 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.125149 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.127732 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.127732 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.128451 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.128751 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.131018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.132969 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.134205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.136703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.137536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.137843 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.138135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.138535 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.138881 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.139189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.140037 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.140342 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.140631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.140916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.141203 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.141490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.155809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.156134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.156409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.156678 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.158399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.158694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.158981 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.159250 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.163367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.167828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.167936 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.168014 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.168014 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.168748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.170292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.170385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.170573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.170655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.170857 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.171169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.171470 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.173418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.173707 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.173998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.174283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.174677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.182797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.183101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.183390 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.183678 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.183965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.184276 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.184571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.186277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.186567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.186853 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.187165 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.187547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.187839 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.188122 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.200876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.203952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.204501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.206898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.207484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.208388 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.209854 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.210144 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.210356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.210484 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.211933 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.213535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.216168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.216168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.217164 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.218697 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.221007 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.221560 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.221858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.222149 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.222432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.222526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.225400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.225798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.226086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.226368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.231233 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.231530 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.231814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.235277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.235573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.235856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.238801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81479) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.239120 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.239407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.239686 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.239962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.240236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.240514 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.240794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.241078 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.241364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.241680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.241949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.242218 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.242489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81479) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.242762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 06 07:52:31.243074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.243409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.243656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.243932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.244204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.244478 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.244762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.245034 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.246419 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.246719 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.247039 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.255471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.255471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 06 07:52:31.277673 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555cb43049c0 pid: 81479 (default app) Oct 06 07:52:31.281153 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e36546-d8a3-4e16-9bed-ac994b416406 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:31.281445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e36546-d8a3-4e16-9bed-ac994b416406 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:31.281718 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e36546-d8a3-4e16-9bed-ac994b416406 None None] PATH_INFO: `/v3` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:31.282487 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555cb43049c0 pid: 81480 (default app) Oct 06 07:52:31.285955 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 791 bytes} [Thu Oct 6 07:52:31 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:52:39.007420 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cc65a-0594-4388-ad4b-d5ba6c8af0cb None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:39.007945 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cc65a-0594-4388-ad4b-d5ba6c8af0cb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:39.007945 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cc65a-0594-4388-ad4b-d5ba6c8af0cb None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:39.012282 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1/2] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:52:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:52:39.017843 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8716921-9bab-447b-a4fe-473ee8a8b7a2 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:39.018272 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8716921-9bab-447b-a4fe-473ee8a8b7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:39.018368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8716921-9bab-447b-a4fe-473ee8a8b7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:39.087695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: INFO dbcounter [None req-a8716921-9bab-447b-a4fe-473ee8a8b7a2 None None] Registered counter for database keystone Oct 06 07:52:39.109533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG oslo_db.sqlalchemy.engines [None req-a8716921-9bab-447b-a4fe-473ee8a8b7a2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81479) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 06 07:52:39.115706 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writer thread running {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 06 07:52:39.295211 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG passlib.handlers.bcrypt [None req-a8716921-9bab-447b-a4fe-473ee8a8b7a2 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81479) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 06 07:52:39.295504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG passlib.handlers.bcrypt [None req-a8716921-9bab-447b-a4fe-473ee8a8b7a2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81479) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 06 07:52:39.352945 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-a8716921-9bab-447b-a4fe-473ee8a8b7a2 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:52:39.355170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a8716921-9bab-447b-a4fe-473ee8a8b7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:52:39.482084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 2/3] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:52:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 06 07:52:39.914362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cc65a-0594-4388-ad4b-d5ba6c8af0cb None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:39.985383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: INFO dbcounter [None req-990cc65a-0594-4388-ad4b-d5ba6c8af0cb None None] Registered counter for database keystone Oct 06 07:52:40.004938 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG oslo_db.sqlalchemy.engines [None req-990cc65a-0594-4388-ad4b-d5ba6c8af0cb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81480) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 06 07:52:40.011670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writer thread running {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 06 07:52:40.129594 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cc65a-0594-4388-ad4b-d5ba6c8af0cb None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:40.133976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d3717-1a65-4067-b851-59480f86b351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:40.134593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d3717-1a65-4067-b851-59480f86b351 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:40.134975 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d3717-1a65-4067-b851-59480f86b351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:40.135252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d3717-1a65-4067-b851-59480f86b351 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:40.144510 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d3717-1a65-4067-b851-59480f86b351 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:40.196787 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6f1d3717-1a65-4067-b851-59480f86b351 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 06 07:52:40.197116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d3717-1a65-4067-b851-59480f86b351 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:40.207818 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 2/4] 10.222.0.30 () {58 vars in 1324 bytes} [Thu Oct 6 07:52:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 07:52:40.214052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8716921-9bab-447b-a4fe-473ee8a8b7a2 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:40.231556 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8716921-9bab-447b-a4fe-473ee8a8b7a2 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:40.236027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d4ae6-0e1a-4116-82f9-ea4f9b2116c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:40.237115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d4ae6-0e1a-4116-82f9-ea4f9b2116c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:40.237396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d4ae6-0e1a-4116-82f9-ea4f9b2116c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:40.237668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d4ae6-0e1a-4116-82f9-ea4f9b2116c2 admin admin] PATH_INFO: `/v3/projects/f2d30f50f03441f1a79068b287cb95c1` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:40.239285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d4ae6-0e1a-4116-82f9-ea4f9b2116c2 admin admin] RBAC: Authorizing `identity:get_project(project_id=f2d30f50f03441f1a79068b287cb95c1)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:40.292526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e74d4ae6-0e1a-4116-82f9-ea4f9b2116c2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 06 07:52:40.292871 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d4ae6-0e1a-4116-82f9-ea4f9b2116c2 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:40.297444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 3/5] 10.222.0.30 () {56 vars in 1207 bytes} [Thu Oct 6 07:52:40 2022] GET /identity/v3/projects/f2d30f50f03441f1a79068b287cb95c1 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:52:42.458379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325e951-5426-48b7-aa23-95b19dcb8a16 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:42.458912 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325e951-5426-48b7-aa23-95b19dcb8a16 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:42.458912 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325e951-5426-48b7-aa23-95b19dcb8a16 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:42.462979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 3/6] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:52:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:52:42.468578 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedad3f-1b66-4b1c-b953-8d292b18acd0 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:42.468954 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedad3f-1b66-4b1c-b953-8d292b18acd0 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:42.469226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedad3f-1b66-4b1c-b953-8d292b18acd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:42.489640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-9eedad3f-1b66-4b1c-b953-8d292b18acd0 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:52:42.491198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9eedad3f-1b66-4b1c-b953-8d292b18acd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:52:42.500198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 4/7] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:52:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 06 07:52:42.769282 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325e951-5426-48b7-aa23-95b19dcb8a16 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:42.781596 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325e951-5426-48b7-aa23-95b19dcb8a16 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:42.785848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48098d9-407e-4105-9bbd-e12aaf2223bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:42.786445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48098d9-407e-4105-9bbd-e12aaf2223bd admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:42.786723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48098d9-407e-4105-9bbd-e12aaf2223bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:42.787021 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48098d9-407e-4105-9bbd-e12aaf2223bd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:42.795226 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48098d9-407e-4105-9bbd-e12aaf2223bd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:42.796998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48098d9-407e-4105-9bbd-e12aaf2223bd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:42.805640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 4/8] 10.222.0.30 () {58 vars in 1324 bytes} [Thu Oct 6 07:52:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 07:52:42.811537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedad3f-1b66-4b1c-b953-8d292b18acd0 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:42.820548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedad3f-1b66-4b1c-b953-8d292b18acd0 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:42.825375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768255fe-5952-44ee-9d7b-ce12f172d494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:42.825992 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768255fe-5952-44ee-9d7b-ce12f172d494 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:42.826269 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768255fe-5952-44ee-9d7b-ce12f172d494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:42.826539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768255fe-5952-44ee-9d7b-ce12f172d494 admin admin] PATH_INFO: `/v3/users/93747f497a1f4adcb4b8ddb03adfe01c` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:42.827880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768255fe-5952-44ee-9d7b-ce12f172d494 admin admin] RBAC: Authorizing `identity:get_user(user_id=93747f497a1f4adcb4b8ddb03adfe01c)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:42.829718 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768255fe-5952-44ee-9d7b-ce12f172d494 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:42.831258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 5/9] 10.222.0.30 () {56 vars in 1198 bytes} [Thu Oct 6 07:52:42 2022] GET /identity/v3/users/93747f497a1f4adcb4b8ddb03adfe01c => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:52:45.096844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f92a2-60a2-4b45-9260-5a9f198bc50f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:45.097383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f92a2-60a2-4b45-9260-5a9f198bc50f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:45.097480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f92a2-60a2-4b45-9260-5a9f198bc50f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:45.098755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 5/10] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:52:45.104651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d11b72-bb2d-4ad5-a125-3d817342f176 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:45.105190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d11b72-bb2d-4ad5-a125-3d817342f176 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:45.105476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d11b72-bb2d-4ad5-a125-3d817342f176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:45.123657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-f6d11b72-bb2d-4ad5-a125-3d817342f176 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:52:45.124844 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f6d11b72-bb2d-4ad5-a125-3d817342f176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:52:45.152825 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae036da1-1ecc-45d8-b0ec-aa7598769389 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:45.153109 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae036da1-1ecc-45d8-b0ec-aa7598769389 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:45.153385 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae036da1-1ecc-45d8-b0ec-aa7598769389 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:45.154884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 6/11] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:52:45.159936 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f7ad2-78b9-4b83-8929-c3a0f9b9bfaf None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:45.160218 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f7ad2-78b9-4b83-8929-c3a0f9b9bfaf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:45.160487 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f7ad2-78b9-4b83-8929-c3a0f9b9bfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:45.201973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG passlib.handlers.bcrypt [None req-456f7ad2-78b9-4b83-8929-c3a0f9b9bfaf None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81480) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 06 07:52:45.202357 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG passlib.handlers.bcrypt [None req-456f7ad2-78b9-4b83-8929-c3a0f9b9bfaf None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81480) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 06 07:52:45.206575 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 6/12] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:52:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:52:45.252238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-456f7ad2-78b9-4b83-8929-c3a0f9b9bfaf None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:52:45.253809 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-456f7ad2-78b9-4b83-8929-c3a0f9b9bfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:52:45.351688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 7/13] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:52:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:52:45.471624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d11b72-bb2d-4ad5-a125-3d817342f176 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:45.495448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:45.575081 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d11b72-bb2d-4ad5-a125-3d817342f176 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:45.608530 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01491df3-6473-4e9f-9e20-75b0a858f0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:45.609224 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01491df3-6473-4e9f-9e20-75b0a858f0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:45.609572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01491df3-6473-4e9f-9e20-75b0a858f0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:45.609931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01491df3-6473-4e9f-9e20-75b0a858f0f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:45.614306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f7ad2-78b9-4b83-8929-c3a0f9b9bfaf None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:45.622159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01491df3-6473-4e9f-9e20-75b0a858f0f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:52:45.631264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-01491df3-6473-4e9f-9e20-75b0a858f0f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:52:45.631264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:52:45.631264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:52:45.631264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:52:45.631264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:52:45.631264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:52:45.631264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:52:45.631264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:52:45.631264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:52:45.631264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:52:45.631264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:52:45.631264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:52:45.631264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:52:45.631264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:52:45.631264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:52:45.631264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:52:45.632240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:52:45.632240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:52:45.632240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:52:45.632240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:52:45.632240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:52:45.632240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:52:45.632240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:52:45.632240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:52:45.632240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:52:45.632240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:52:45.632240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:52:45.632240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:52:45.632240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:52:45.632240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:52:45.632240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:52:45.632240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:52:45.633150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:52:45.633150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:52:45.633150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:52:45.633150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:52:45.633150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:52:45.633150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:52:45.633150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:52:45.633150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:52:45.633150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:52:45.633150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:52:45.633150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:52:45.633150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:52:45.633150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 7/14] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:52:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:52:45.639036 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01491df3-6473-4e9f-9e20-75b0a858f0f9 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:45.720851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:45.727657 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f7ad2-78b9-4b83-8929-c3a0f9b9bfaf None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:45.730996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:45.743101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01491df3-6473-4e9f-9e20-75b0a858f0f9 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:45.767652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6f288-682b-4ecc-83ae-f101ec57fac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:45.768523 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6f288-682b-4ecc-83ae-f101ec57fac5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:45.769398 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6f288-682b-4ecc-83ae-f101ec57fac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:45.769557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6f288-682b-4ecc-83ae-f101ec57fac5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:45.774722 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c0671-9581-4fc1-b5fc-eb0abff848e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:45.775364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c0671-9581-4fc1-b5fc-eb0abff848e7 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:45.775645 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c0671-9581-4fc1-b5fc-eb0abff848e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:45.776588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c0671-9581-4fc1-b5fc-eb0abff848e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:45.777223 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c0671-9581-4fc1-b5fc-eb0abff848e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:52:45.777696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c0671-9581-4fc1-b5fc-eb0abff848e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:45.779226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c0671-9581-4fc1-b5fc-eb0abff848e7 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:45.791236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 8/15] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:52:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:52:45.797535 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c0671-9581-4fc1-b5fc-eb0abff848e7 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:45.842761 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6f288-682b-4ecc-83ae-f101ec57fac5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:45.844327 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6f288-682b-4ecc-83ae-f101ec57fac5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:45.897439 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c0671-9581-4fc1-b5fc-eb0abff848e7 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:45.932945 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724d281-42ee-44ef-adfd-516af8ed9f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:45.933670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724d281-42ee-44ef-adfd-516af8ed9f88 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:45.933948 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724d281-42ee-44ef-adfd-516af8ed9f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:45.934226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724d281-42ee-44ef-adfd-516af8ed9f88 None admin] PATH_INFO: `/v3/users/93747f497a1f4adcb4b8ddb03adfe01c` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:45.935392 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724d281-42ee-44ef-adfd-516af8ed9f88 None admin] RBAC: Authorizing `identity:get_user(user_id=93747f497a1f4adcb4b8ddb03adfe01c)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:45.936855 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724d281-42ee-44ef-adfd-516af8ed9f88 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:45.938417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 9/16] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:52:45 2022] GET /identity/v3/users/93747f497a1f4adcb4b8ddb03adfe01c => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:52:45.942284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 8/17] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:52:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 07:52:45.943739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724d281-42ee-44ef-adfd-516af8ed9f88 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:45.948300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6f288-682b-4ecc-83ae-f101ec57fac5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:45.951171 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:45.958523 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:46.042719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724d281-42ee-44ef-adfd-516af8ed9f88 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:46.044963 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6f288-682b-4ecc-83ae-f101ec57fac5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:46.077679 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794643f-9119-4adc-8a8e-b65819b11426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:46.078300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794643f-9119-4adc-8a8e-b65819b11426 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:46.078575 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794643f-9119-4adc-8a8e-b65819b11426 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:46.078844 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794643f-9119-4adc-8a8e-b65819b11426 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:46.079153 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eca39c-ca11-4d0d-9723-80834546faf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:46.079966 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794643f-9119-4adc-8a8e-b65819b11426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:46.080057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eca39c-ca11-4d0d-9723-80834546faf1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:46.080270 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eca39c-ca11-4d0d-9723-80834546faf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:46.080542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eca39c-ca11-4d0d-9723-80834546faf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:46.081456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794643f-9119-4adc-8a8e-b65819b11426 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:46.082992 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 10/18] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:52:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:52:46.088899 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794643f-9119-4adc-8a8e-b65819b11426 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:46.099930 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eca39c-ca11-4d0d-9723-80834546faf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:46.101415 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eca39c-ca11-4d0d-9723-80834546faf1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:46.120504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-73eca39c-ca11-4d0d-9723-80834546faf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:52:46.120504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:52:46.120504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:52:46.120504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:52:46.120504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:52:46.120504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:52:46.120504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:52:46.120504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:52:46.120504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:52:46.120504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:52:46.120504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:52:46.120504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:52:46.120504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:52:46.120504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:52:46.120504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:52:46.120504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:52:46.126300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:52:46.127338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:52:46.127338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:52:46.127338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:52:46.127338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 9/19] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:52:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:52:46.128041 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eca39c-ca11-4d0d-9723-80834546faf1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:46.168264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:46.187031 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794643f-9119-4adc-8a8e-b65819b11426 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:46.190691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:46.223075 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c89848-bdbc-487d-bd2f-b0299ff1bd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:46.223685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c89848-bdbc-487d-bd2f-b0299ff1bd52 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:46.223962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c89848-bdbc-487d-bd2f-b0299ff1bd52 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:46.224239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c89848-bdbc-487d-bd2f-b0299ff1bd52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:46.225034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c89848-bdbc-487d-bd2f-b0299ff1bd52 None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.domain.id=default, user.id=93747f497a1f4adcb4b8ddb03adfe01c) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:52:46.225339 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c89848-bdbc-487d-bd2f-b0299ff1bd52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:46.226776 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c89848-bdbc-487d-bd2f-b0299ff1bd52 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:46.233860 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 11/20] 10.222.0.30 () {56 vars in 1322 bytes} [Thu Oct 6 07:52:46 2022] GET /identity/v3/role_assignments?user.id=93747f497a1f4adcb4b8ddb03adfe01c&scope.domain.id=default&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:52:46.235500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eca39c-ca11-4d0d-9723-80834546faf1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:46.272815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac8b3b-77e8-434b-baa0-1be2f03ff0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:46.273440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac8b3b-77e8-434b-baa0-1be2f03ff0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:46.273718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac8b3b-77e8-434b-baa0-1be2f03ff0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:46.273988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac8b3b-77e8-434b-baa0-1be2f03ff0f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:46.274649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac8b3b-77e8-434b-baa0-1be2f03ff0f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:52:46.275009 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac8b3b-77e8-434b-baa0-1be2f03ff0f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:46.276435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac8b3b-77e8-434b-baa0-1be2f03ff0f4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:46.296460 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 10/21] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:52:48.826992 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e2b5f-658c-4426-bea5-6550a95ad5eb None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:48.828038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e2b5f-658c-4426-bea5-6550a95ad5eb None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:48.828517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e2b5f-658c-4426-bea5-6550a95ad5eb None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:48.830074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 12/22] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:52:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:52:48.840038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40314089-2af0-486f-8917-e3086fe4e10b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:48.840867 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40314089-2af0-486f-8917-e3086fe4e10b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:48.841375 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40314089-2af0-486f-8917-e3086fe4e10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:48.868183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-40314089-2af0-486f-8917-e3086fe4e10b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:52:48.870730 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-40314089-2af0-486f-8917-e3086fe4e10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:52:48.957796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 11/23] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:52:49.181929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d1579-f61e-4948-a340-78ade04d8a28 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:49.183151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d1579-f61e-4948-a340-78ade04d8a28 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:49.183640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d1579-f61e-4948-a340-78ade04d8a28 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:49.185513 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 13/24] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:52:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:52:49.190579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1762e5b-839d-4234-b4ea-e10aadaf54e7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:49.191126 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1762e5b-839d-4234-b4ea-e10aadaf54e7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:49.191613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1762e5b-839d-4234-b4ea-e10aadaf54e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:49.217951 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a1762e5b-839d-4234-b4ea-e10aadaf54e7 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:52:49.219488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a1762e5b-839d-4234-b4ea-e10aadaf54e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:52:49.289399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:49.330081 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 12/25] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:52:49.446663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72557fc7-6daf-40d9-8b50-e024bb6e1e54 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:49.447324 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72557fc7-6daf-40d9-8b50-e024bb6e1e54 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:49.449078 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72557fc7-6daf-40d9-8b50-e024bb6e1e54 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:49.450774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 14/26] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:52:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:52:49.457095 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7b8e8-ad49-4f58-8965-4e548ab935b8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:49.457700 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7b8e8-ad49-4f58-8965-4e548ab935b8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:49.458204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7b8e8-ad49-4f58-8965-4e548ab935b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:49.485017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c0a7b8e8-ad49-4f58-8965-4e548ab935b8 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:52:49.486520 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c0a7b8e8-ad49-4f58-8965-4e548ab935b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:52:49.546681 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72557fc7-6daf-40d9-8b50-e024bb6e1e54 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:49.620291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 13/27] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:52:49.622636 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d09959-3c42-4486-ae72-4b980483c16d None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:49.623773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d09959-3c42-4486-ae72-4b980483c16d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:49.625423 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d09959-3c42-4486-ae72-4b980483c16d None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:49.627938 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 14/28] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:52:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:52:49.634465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c5835-b804-49f2-b766-fa8dda5e4018 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:49.639321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c5835-b804-49f2-b766-fa8dda5e4018 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:49.641055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c5835-b804-49f2-b766-fa8dda5e4018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:49.676625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-289c5835-b804-49f2-b766-fa8dda5e4018 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:52:49.678620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-289c5835-b804-49f2-b766-fa8dda5e4018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:52:49.764603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 15/29] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:52:49.765970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c5835-b804-49f2-b766-fa8dda5e4018 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:49.803830 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:49.810281 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72557fc7-6daf-40d9-8b50-e024bb6e1e54 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:49.877136 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb16e2-6fdf-420f-ac36-7e03636cd640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:49.877751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb16e2-6fdf-420f-ac36-7e03636cd640 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:49.878034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb16e2-6fdf-420f-ac36-7e03636cd640 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:49.878303 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb16e2-6fdf-420f-ac36-7e03636cd640 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:49.900010 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb16e2-6fdf-420f-ac36-7e03636cd640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:49.901749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb16e2-6fdf-420f-ac36-7e03636cd640 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:49.905017 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:49.911441 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c5835-b804-49f2-b766-fa8dda5e4018 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:49.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-cfcb16e2-6fdf-420f-ac36-7e03636cd640 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:52:49.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:52:49.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:52:49.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:52:49.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:52:49.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:52:49.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:52:49.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:52:49.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:52:49.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:52:49.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:52:49.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:52:49.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:52:49.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:52:49.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:52:49.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:52:49.923635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:52:49.924438 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:52:49.924438 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:52:49.924438 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:52:49.927833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 15/30] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:52:49.930338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb16e2-6fdf-420f-ac36-7e03636cd640 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:49.955550 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0567d-d3f3-42fe-bff7-f610c1af0d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:49.957074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0567d-d3f3-42fe-bff7-f610c1af0d22 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:49.957747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0567d-d3f3-42fe-bff7-f610c1af0d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:49.958406 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0567d-d3f3-42fe-bff7-f610c1af0d22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:49.976519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0567d-d3f3-42fe-bff7-f610c1af0d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:52:49.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d0d0567d-d3f3-42fe-bff7-f610c1af0d22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:52:49.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:52:49.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:52:49.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:52:49.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:52:49.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:52:49.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:52:49.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:52:49.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:52:49.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:52:49.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:52:49.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:52:49.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:52:49.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:52:49.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:52:49.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:52:49.990931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:52:49.990931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:52:49.990931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:52:49.990931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:52:49.990931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:52:49.990931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:52:49.990931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:52:49.990931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:52:49.990931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:52:49.990931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:52:49.990931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:52:49.990931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:52:49.990931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:52:49.990931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:52:49.990931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:52:49.990931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:52:49.991746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:52:49.991746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:52:49.991746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:52:49.991746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:52:49.991746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:52:49.991746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:52:49.991746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:52:49.991746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:52:49.991746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:52:49.991746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:52:49.991746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:52:49.991746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:52:49.995986 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 16/31] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:52:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:52:49.997225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0567d-d3f3-42fe-bff7-f610c1af0d22 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:50.108735 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:50.114888 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0567d-d3f3-42fe-bff7-f610c1af0d22 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:50.116197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb16e2-6fdf-420f-ac36-7e03636cd640 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:50.154813 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc0e93-29ae-4370-be56-1dba160a7991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:50.155465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc0e93-29ae-4370-be56-1dba160a7991 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:50.155743 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc0e93-29ae-4370-be56-1dba160a7991 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:50.156016 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc0e93-29ae-4370-be56-1dba160a7991 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:50.156900 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc0e93-29ae-4370-be56-1dba160a7991 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:50.158408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc0e93-29ae-4370-be56-1dba160a7991 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:50.177668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e5db9-5fc6-4a54-857a-da4f3f866c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:50.178278 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e5db9-5fc6-4a54-857a-da4f3f866c87 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:50.178553 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e5db9-5fc6-4a54-857a-da4f3f866c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:50.178825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e5db9-5fc6-4a54-857a-da4f3f866c87 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:50.179498 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e5db9-5fc6-4a54-857a-da4f3f866c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:52:50.179798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e5db9-5fc6-4a54-857a-da4f3f866c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:50.181188 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e5db9-5fc6-4a54-857a-da4f3f866c87 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:50.200907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 16/32] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:52:50.202321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e5db9-5fc6-4a54-857a-da4f3f866c87 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:50.266632 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:50.320381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e5db9-5fc6-4a54-857a-da4f3f866c87 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:50.361063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270dcf9-6700-4152-bb45-0f9c26b09321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:50.361771 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270dcf9-6700-4152-bb45-0f9c26b09321 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:50.362064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270dcf9-6700-4152-bb45-0f9c26b09321 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:50.362412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270dcf9-6700-4152-bb45-0f9c26b09321 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:50.363247 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270dcf9-6700-4152-bb45-0f9c26b09321 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:52:50.363629 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270dcf9-6700-4152-bb45-0f9c26b09321 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:50.365202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270dcf9-6700-4152-bb45-0f9c26b09321 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:50.366698 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 17/33] 10.222.0.30 () {60 vars in 1131 bytes} [Thu Oct 6 07:52:49 2022] POST /identity/v3/roles => generated 225 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:52:50.367766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc0e93-29ae-4370-be56-1dba160a7991 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:50.378078 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 17/34] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:52:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:52:50.379309 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270dcf9-6700-4152-bb45-0f9c26b09321 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:50.513693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270dcf9-6700-4152-bb45-0f9c26b09321 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:50.538775 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:50.561466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aabc9b-ef5a-4017-8979-9b10ba6223e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:50.562378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aabc9b-ef5a-4017-8979-9b10ba6223e5 None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:50.562825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aabc9b-ef5a-4017-8979-9b10ba6223e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:50.563138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc0e93-29ae-4370-be56-1dba160a7991 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:50.563440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aabc9b-ef5a-4017-8979-9b10ba6223e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:50.564265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aabc9b-ef5a-4017-8979-9b10ba6223e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:50.566220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aabc9b-ef5a-4017-8979-9b10ba6223e5 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:50.601570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b94654-31d4-4472-9a1e-f71294586202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:50.602193 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b94654-31d4-4472-9a1e-f71294586202 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:50.602474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b94654-31d4-4472-9a1e-f71294586202 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:50.602749 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b94654-31d4-4472-9a1e-f71294586202 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:50.603495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b94654-31d4-4472-9a1e-f71294586202 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:50.605822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b94654-31d4-4472-9a1e-f71294586202 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:50.695913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 18/35] 10.222.0.30 () {60 vars in 1131 bytes} [Thu Oct 6 07:52:50 2022] POST /identity/v3/roles => generated 219 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:52:50.695913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b94654-31d4-4472-9a1e-f71294586202 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:50.719163 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:50.784477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 18/36] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Oct 6 07:52:50 2022] POST /identity/v3/projects => generated 301 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:52:50.784477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce06c72-6623-4c8a-8331-9e4e489807de None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:50.784477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce06c72-6623-4c8a-8331-9e4e489807de None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:50.784477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce06c72-6623-4c8a-8331-9e4e489807de None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:50.784477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 19/37] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:52:50.784477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158063d-c495-411c-b9ec-11eaa619c67b None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:50.784477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158063d-c495-411c-b9ec-11eaa619c67b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:50.785056 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158063d-c495-411c-b9ec-11eaa619c67b None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:50.785056 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 20/38] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:52:50.788600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212eb08-94cc-4efb-82c7-e3d94c3ad3c7 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:50.788906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212eb08-94cc-4efb-82c7-e3d94c3ad3c7 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:50.789155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212eb08-94cc-4efb-82c7-e3d94c3ad3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:50.817373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-7212eb08-94cc-4efb-82c7-e3d94c3ad3c7 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:52:50.819687 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7212eb08-94cc-4efb-82c7-e3d94c3ad3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:52:50.827855 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b94654-31d4-4472-9a1e-f71294586202 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:50.870808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa072ed4-eade-44d5-82b8-32c88419706e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:50.871493 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa072ed4-eade-44d5-82b8-32c88419706e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:50.871772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa072ed4-eade-44d5-82b8-32c88419706e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:50.872045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa072ed4-eade-44d5-82b8-32c88419706e None admin] PATH_INFO: `/v3/users/93747f497a1f4adcb4b8ddb03adfe01c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:50.874195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa072ed4-eade-44d5-82b8-32c88419706e None admin] RBAC: Authorizing `identity:get_user(user_id=93747f497a1f4adcb4b8ddb03adfe01c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:50.875788 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa072ed4-eade-44d5-82b8-32c88419706e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:50.877656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 19/39] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:52:50 2022] GET /identity/v3/users/93747f497a1f4adcb4b8ddb03adfe01c => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:52:50.883828 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1aae7-e50f-41e1-a9d9-20c65d38490a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:50.884121 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1aae7-e50f-41e1-a9d9-20c65d38490a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:50.884380 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1aae7-e50f-41e1-a9d9-20c65d38490a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:50.907911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f9e1aae7-e50f-41e1-a9d9-20c65d38490a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:52:50.909732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f9e1aae7-e50f-41e1-a9d9-20c65d38490a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:52:50.924183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 21/40] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:52:50.925077 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212eb08-94cc-4efb-82c7-e3d94c3ad3c7 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:51.000840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 20/41] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:52:51.076113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:51.081598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212eb08-94cc-4efb-82c7-e3d94c3ad3c7 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:51.118371 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fd2f3-8bd1-4d8d-8a1f-6589e051ab51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:51.119063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fd2f3-8bd1-4d8d-8a1f-6589e051ab51 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:51.120107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fd2f3-8bd1-4d8d-8a1f-6589e051ab51 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:51.120384 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fd2f3-8bd1-4d8d-8a1f-6589e051ab51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:51.121508 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fd2f3-8bd1-4d8d-8a1f-6589e051ab51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:51.123143 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fd2f3-8bd1-4d8d-8a1f-6589e051ab51 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:51.125163 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 22/42] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:52:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:52:51.130660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1aae7-e50f-41e1-a9d9-20c65d38490a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:51.153480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:51.219821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fd2f3-8bd1-4d8d-8a1f-6589e051ab51 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:51.235974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1aae7-e50f-41e1-a9d9-20c65d38490a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:51.271753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1bee0f-ae4c-4a6c-9556-646abb28722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:51.272334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bee0f-ae4c-4a6c-9556-646abb28722c None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:51.272616 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bee0f-ae4c-4a6c-9556-646abb28722c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:51.272893 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bee0f-ae4c-4a6c-9556-646abb28722c None admin] PATH_INFO: `/v3/domains/default/users/93747f497a1f4adcb4b8ddb03adfe01c/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:51.274823 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1bee0f-ae4c-4a6c-9556-646abb28722c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=93747f497a1f4adcb4b8ddb03adfe01c, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:51.276504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1bee0f-ae4c-4a6c-9556-646abb28722c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:51.327987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fd2f3-8bd1-4d8d-8a1f-6589e051ab51 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:51.362863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 21/43] 10.222.0.30 () {58 vars in 1361 bytes} [Thu Oct 6 07:52:51 2022] PUT /identity/v3/domains/default/users/93747f497a1f4adcb4b8ddb03adfe01c/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:52:51.363769 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1bee0f-ae4c-4a6c-9556-646abb28722c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:51.454372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:51.479407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12799f64-ac84-486c-8fcd-e16e84ecc4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:51.480222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12799f64-ac84-486c-8fcd-e16e84ecc4e9 None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:51.480485 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12799f64-ac84-486c-8fcd-e16e84ecc4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:51.480763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12799f64-ac84-486c-8fcd-e16e84ecc4e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:51.481463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12799f64-ac84-486c-8fcd-e16e84ecc4e9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:51.482963 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12799f64-ac84-486c-8fcd-e16e84ecc4e9 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:51.483158 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1bee0f-ae4c-4a6c-9556-646abb28722c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:51.521411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29baa278-ca6b-4546-b2a1-8d9c34c09c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:51.522103 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baa278-ca6b-4546-b2a1-8d9c34c09c6f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:51.522330 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baa278-ca6b-4546-b2a1-8d9c34c09c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:51.522605 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baa278-ca6b-4546-b2a1-8d9c34c09c6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:51.523777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29baa278-ca6b-4546-b2a1-8d9c34c09c6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:51.525271 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29baa278-ca6b-4546-b2a1-8d9c34c09c6f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:51.527059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 22/44] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:52:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:52:51.532608 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29baa278-ca6b-4546-b2a1-8d9c34c09c6f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:51.580524 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 23/45] 10.222.0.30 () {60 vars in 1131 bytes} [Thu Oct 6 07:52:51 2022] POST /identity/v3/roles => generated 223 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 07:52:51.757680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29baa278-ca6b-4546-b2a1-8d9c34c09c6f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:51.794856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfff19f-8b72-4fd4-8ed6-7f38f30cad66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:51.795459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfff19f-8b72-4fd4-8ed6-7f38f30cad66 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:51.795739 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfff19f-8b72-4fd4-8ed6-7f38f30cad66 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:51.796005 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfff19f-8b72-4fd4-8ed6-7f38f30cad66 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:51.796661 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfff19f-8b72-4fd4-8ed6-7f38f30cad66 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:51.798114 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfff19f-8b72-4fd4-8ed6-7f38f30cad66 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:51.815115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:51.889888 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 23/46] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Oct 6 07:52:51 2022] POST /identity/v3/projects => generated 312 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 07:52:53.523008 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a042c96-b35a-4da1-8c75-2e07431b923a None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:53.523522 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a042c96-b35a-4da1-8c75-2e07431b923a None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:53.523637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a042c96-b35a-4da1-8c75-2e07431b923a None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:53.524935 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 24/47] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:52:53.530359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5582d4a-dbd4-43d2-8062-2754adcac0b7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:53.530768 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5582d4a-dbd4-43d2-8062-2754adcac0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:53.531060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5582d4a-dbd4-43d2-8062-2754adcac0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:53.549060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b5582d4a-dbd4-43d2-8062-2754adcac0b7 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:52:53.550243 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b5582d4a-dbd4-43d2-8062-2754adcac0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:52:53.615374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 24/48] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:52:53.881493 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a042c96-b35a-4da1-8c75-2e07431b923a None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:53.911240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:53.988909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a042c96-b35a-4da1-8c75-2e07431b923a None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:54.022595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bf16b-ca07-4253-a70a-5cf171c864dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:54.023263 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bf16b-ca07-4253-a70a-5cf171c864dd None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:54.023548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bf16b-ca07-4253-a70a-5cf171c864dd None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:54.023826 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bf16b-ca07-4253-a70a-5cf171c864dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:54.035069 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bf16b-ca07-4253-a70a-5cf171c864dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:52:54.036015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e7538-00c2-4d8e-abed-44bfb766cbb7 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:54.036303 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e7538-00c2-4d8e-abed-44bfb766cbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:54.036574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e7538-00c2-4d8e-abed-44bfb766cbb7 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:54.037926 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 25/49] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:52:54.040540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-3c2bf16b-ca07-4253-a70a-5cf171c864dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:52:54.040540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:52:54.040540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:52:54.040540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:52:54.040540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:52:54.040540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:52:54.040540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:52:54.040540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:52:54.040540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:52:54.040540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:52:54.040540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:52:54.040540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:52:54.040540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:52:54.040540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:52:54.040540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:52:54.040540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:52:54.041636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:52:54.041636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:52:54.041636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:52:54.041636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:52:54.041636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:52:54.041636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:52:54.041636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:52:54.041636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:52:54.041636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:52:54.041636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:52:54.041636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:52:54.041636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:52:54.041636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:52:54.041636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:52:54.041636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:52:54.041636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:52:54.042401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:52:54.042401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:52:54.042401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:52:54.042401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:52:54.042401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:52:54.042401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:52:54.042401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:52:54.042401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:52:54.042401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:52:54.042401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:52:54.042401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:52:54.042401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:52:54.042401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 25/50] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:52:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:52:54.043639 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963b2b7-dc24-42d2-a8ff-af2152140da7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:54.043923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963b2b7-dc24-42d2-a8ff-af2152140da7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:54.044193 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963b2b7-dc24-42d2-a8ff-af2152140da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:54.047583 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bf16b-ca07-4253-a70a-5cf171c864dd None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:54.061900 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f963b2b7-dc24-42d2-a8ff-af2152140da7 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:52:54.063110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f963b2b7-dc24-42d2-a8ff-af2152140da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:52:54.131616 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 26/51] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:52:54.137654 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:54.149563 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bf16b-ca07-4253-a70a-5cf171c864dd None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:54.184849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee001bd-e590-4206-95b9-ad76c9cd0da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:54.185452 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee001bd-e590-4206-95b9-ad76c9cd0da0 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:54.185731 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee001bd-e590-4206-95b9-ad76c9cd0da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:54.186002 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee001bd-e590-4206-95b9-ad76c9cd0da0 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:54.186663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee001bd-e590-4206-95b9-ad76c9cd0da0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:52:54.187027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee001bd-e590-4206-95b9-ad76c9cd0da0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:54.188420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee001bd-e590-4206-95b9-ad76c9cd0da0 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:54.196607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 26/52] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:52:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:52:54.202848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963b2b7-dc24-42d2-a8ff-af2152140da7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:54.257003 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:54.300118 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963b2b7-dc24-42d2-a8ff-af2152140da7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:54.334072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cfee1-13fe-4006-b8af-b8cbcffaedd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:54.334775 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cfee1-13fe-4006-b8af-b8cbcffaedd6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:54.335142 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cfee1-13fe-4006-b8af-b8cbcffaedd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:54.335419 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cfee1-13fe-4006-b8af-b8cbcffaedd6 None admin] PATH_INFO: `/v3/users/93747f497a1f4adcb4b8ddb03adfe01c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:54.336541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cfee1-13fe-4006-b8af-b8cbcffaedd6 None admin] RBAC: Authorizing `identity:get_user(user_id=93747f497a1f4adcb4b8ddb03adfe01c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:54.338029 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cfee1-13fe-4006-b8af-b8cbcffaedd6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:54.339638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 27/53] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:52:54 2022] GET /identity/v3/users/93747f497a1f4adcb4b8ddb03adfe01c => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:52:54.345112 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee001bd-e590-4206-95b9-ad76c9cd0da0 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:54.405694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cfee1-13fe-4006-b8af-b8cbcffaedd6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:54.445846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee001bd-e590-4206-95b9-ad76c9cd0da0 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:54.479366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad6d6b-b4ef-435e-8528-44f3f9df8623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:54.479999 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad6d6b-b4ef-435e-8528-44f3f9df8623 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:54.480282 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad6d6b-b4ef-435e-8528-44f3f9df8623 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:54.480559 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad6d6b-b4ef-435e-8528-44f3f9df8623 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:54.481709 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad6d6b-b4ef-435e-8528-44f3f9df8623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:54.483225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad6d6b-b4ef-435e-8528-44f3f9df8623 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:54.484795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 27/54] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:52:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:52:54.490408 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad6d6b-b4ef-435e-8528-44f3f9df8623 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:54.505384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cfee1-13fe-4006-b8af-b8cbcffaedd6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:54.509900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:54.527943 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:54.538992 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a86ce0-a5db-4c21-ab2b-f9eca54d6788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:54.539631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a86ce0-a5db-4c21-ab2b-f9eca54d6788 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:54.539914 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a86ce0-a5db-4c21-ab2b-f9eca54d6788 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:54.540190 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a86ce0-a5db-4c21-ab2b-f9eca54d6788 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:54.542073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a86ce0-a5db-4c21-ab2b-f9eca54d6788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:54.543630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a86ce0-a5db-4c21-ab2b-f9eca54d6788 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:54.545203 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 28/55] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:52:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:52:54.550735 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a86ce0-a5db-4c21-ab2b-f9eca54d6788 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:54.589348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad6d6b-b4ef-435e-8528-44f3f9df8623 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:54.625429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ab1c4-4bed-4843-8e2e-77c7dcb946da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:54.626030 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ab1c4-4bed-4843-8e2e-77c7dcb946da None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:54.626314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ab1c4-4bed-4843-8e2e-77c7dcb946da None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:54.626973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ab1c4-4bed-4843-8e2e-77c7dcb946da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:54.627709 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ab1c4-4bed-4843-8e2e-77c7dcb946da None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.domain.id=default, user.id=93747f497a1f4adcb4b8ddb03adfe01c) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:52:54.628330 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ab1c4-4bed-4843-8e2e-77c7dcb946da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:54.629810 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ab1c4-4bed-4843-8e2e-77c7dcb946da None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:54.635855 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 28/56] 10.222.0.30 () {56 vars in 1322 bytes} [Thu Oct 6 07:52:54 2022] GET /identity/v3/role_assignments?user.id=93747f497a1f4adcb4b8ddb03adfe01c&scope.domain.id=default&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:52:54.649486 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a86ce0-a5db-4c21-ab2b-f9eca54d6788 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:54.686628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e174e8-3742-448d-b2e2-e9103bd7a32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:54.687202 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e174e8-3742-448d-b2e2-e9103bd7a32d None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:54.687489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e174e8-3742-448d-b2e2-e9103bd7a32d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:54.687762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e174e8-3742-448d-b2e2-e9103bd7a32d None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:54.688433 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e174e8-3742-448d-b2e2-e9103bd7a32d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:54.689922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e174e8-3742-448d-b2e2-e9103bd7a32d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:54.786641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 29/57] 10.222.0.30 () {60 vars in 1140 bytes} [Thu Oct 6 07:52:54 2022] POST /identity/v3/projects => generated 298 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 07:52:56.967852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7e913-627c-4f12-a806-8b0a79ff3c19 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:56.968490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7e913-627c-4f12-a806-8b0a79ff3c19 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:56.968490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7e913-627c-4f12-a806-8b0a79ff3c19 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:56.969766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 29/58] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:52:56.975353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f714fe4-5634-4ac4-a656-dc1dc9a73416 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:56.976032 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f714fe4-5634-4ac4-a656-dc1dc9a73416 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:56.976032 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f714fe4-5634-4ac4-a656-dc1dc9a73416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:56.996776 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5f714fe4-5634-4ac4-a656-dc1dc9a73416 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:52:56.998812 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5f714fe4-5634-4ac4-a656-dc1dc9a73416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:52:57.044881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:57.075086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 30/59] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:52:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:52:57.350688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7e913-627c-4f12-a806-8b0a79ff3c19 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:57.432869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:52:57.451441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7e913-627c-4f12-a806-8b0a79ff3c19 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:57.484720 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2fe79-882c-4521-8e30-4db24cb6cc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:57.485343 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2fe79-882c-4521-8e30-4db24cb6cc68 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:57.485625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2fe79-882c-4521-8e30-4db24cb6cc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:57.485902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2fe79-882c-4521-8e30-4db24cb6cc68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:57.487049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2fe79-882c-4521-8e30-4db24cb6cc68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:57.488505 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2fe79-882c-4521-8e30-4db24cb6cc68 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:57.490073 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 30/60] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:52:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:52:57.495741 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f714fe4-5634-4ac4-a656-dc1dc9a73416 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:52:57.589348 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f714fe4-5634-4ac4-a656-dc1dc9a73416 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:52:57.622189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c997e-a12a-4433-bdd8-95927e4a70e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:52:57.622721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c997e-a12a-4433-bdd8-95927e4a70e2 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:52:57.623020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c997e-a12a-4433-bdd8-95927e4a70e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:52:57.623291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c997e-a12a-4433-bdd8-95927e4a70e2 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:52:57.624185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c997e-a12a-4433-bdd8-95927e4a70e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:52:57.625606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c997e-a12a-4433-bdd8-95927e4a70e2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:52:57.727388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 31/61] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:52:57 2022] POST /identity/v3/users => generated 275 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:53:01.191504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b2077-0de3-439e-8c78-318968e88009 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:01.191994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b2077-0de3-439e-8c78-318968e88009 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:01.192066 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b2077-0de3-439e-8c78-318968e88009 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:01.193396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 31/62] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:01.199453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d02c0-f817-41fd-a37a-d20775415e44 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:01.200204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d02c0-f817-41fd-a37a-d20775415e44 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:01.200635 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d02c0-f817-41fd-a37a-d20775415e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:01.236334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2c9d02c0-f817-41fd-a37a-d20775415e44 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:01.237795 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2c9d02c0-f817-41fd-a37a-d20775415e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:01.272508 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6396a85-7baf-4f47-be56-a6888be5cff0 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:01.273138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6396a85-7baf-4f47-be56-a6888be5cff0 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:01.273616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6396a85-7baf-4f47-be56-a6888be5cff0 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:01.275200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 32/63] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:53:01.278489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187b652-898c-49b2-b955-e0c7971d1af3 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:01.279006 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187b652-898c-49b2-b955-e0c7971d1af3 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:01.279499 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187b652-898c-49b2-b955-e0c7971d1af3 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:01.280911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 33/64] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:01.285276 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec1c19-493f-4f8d-8297-7c64b93af58e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:01.285836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec1c19-493f-4f8d-8297-7c64b93af58e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:01.286472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec1c19-493f-4f8d-8297-7c64b93af58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:01.287620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:01.307285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-f3ec1c19-493f-4f8d-8297-7c64b93af58e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:01.309268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f3ec1c19-493f-4f8d-8297-7c64b93af58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:01.321376 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 32/65] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:01.323653 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145af97-03ef-4689-b69e-86d88385373a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:01.324061 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145af97-03ef-4689-b69e-86d88385373a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:01.324455 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145af97-03ef-4689-b69e-86d88385373a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:01.345117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3145af97-03ef-4689-b69e-86d88385373a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:01.346614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3145af97-03ef-4689-b69e-86d88385373a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:01.441196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 34/66] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:01.454010 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 33/67] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:01.690822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055da39-8eb7-4bc1-94ac-421105ad1b9e None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:01.690822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055da39-8eb7-4bc1-94ac-421105ad1b9e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:01.690822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055da39-8eb7-4bc1-94ac-421105ad1b9e None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:01.690822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 35/68] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:01.696954 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1192ac-b2fd-4327-8d0d-5db57333f9e0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:01.697989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1192ac-b2fd-4327-8d0d-5db57333f9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:01.698489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1192ac-b2fd-4327-8d0d-5db57333f9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:01.706914 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055da39-8eb7-4bc1-94ac-421105ad1b9e None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:01.723174 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-da1192ac-b2fd-4327-8d0d-5db57333f9e0 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:01.724647 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-da1192ac-b2fd-4327-8d0d-5db57333f9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:01.806452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 34/69] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:01.808172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1192ac-b2fd-4327-8d0d-5db57333f9e0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:01.812991 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:01.830191 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:01.851552 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055da39-8eb7-4bc1-94ac-421105ad1b9e None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:01.873216 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51081f3-b641-4f8e-92e7-3671ba84c5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:01.874097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51081f3-b641-4f8e-92e7-3671ba84c5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:01.874374 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51081f3-b641-4f8e-92e7-3671ba84c5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:01.874653 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51081f3-b641-4f8e-92e7-3671ba84c5d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:01.885386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51081f3-b641-4f8e-92e7-3671ba84c5d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:01.890264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-a51081f3-b641-4f8e-92e7-3671ba84c5d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:01.890264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:01.890264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:01.890264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:01.890264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:01.890264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:01.890264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:01.890264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:01.890264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:01.890264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:01.890264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:01.890264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:01.890264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:01.890264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:01.890264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:01.890264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:01.891170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:01.891170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:01.891170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:01.891170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:01.891170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:01.891170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:01.891170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:01.891170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:01.891170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:01.891170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:01.891170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:01.891170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:01.891170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:01.891170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:01.891170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:01.891170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:01.892005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:01.892005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:01.892005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:01.892005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:01.892005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:01.892005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:01.892005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:01.892005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:01.892005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:01.892005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:01.892005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:01.892005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:01.892005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 36/70] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:53:01.893157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cb47b-3657-4452-8c91-1fb8aa080b7b None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:01.893157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cb47b-3657-4452-8c91-1fb8aa080b7b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:01.893157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cb47b-3657-4452-8c91-1fb8aa080b7b None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:01.894473 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 37/71] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:01.898830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212cb47b-3657-4452-8c91-1fb8aa080b7b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:01.922980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1192ac-b2fd-4327-8d0d-5db57333f9e0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:01.961697 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa6060-2e53-4f3b-99c6-2050620c8442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:01.962246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa6060-2e53-4f3b-99c6-2050620c8442 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:01.962554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa6060-2e53-4f3b-99c6-2050620c8442 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:01.962795 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa6060-2e53-4f3b-99c6-2050620c8442 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:01.975167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa6060-2e53-4f3b-99c6-2050620c8442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:01.980763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1afa6060-2e53-4f3b-99c6-2050620c8442 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:01.980763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:01.980763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:01.980763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:01.980763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:01.980763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:01.980763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:01.980763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:01.980763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:01.980763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:01.980763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:01.980763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:01.980763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:01.980763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:01.980763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:01.980763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:01.981607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:01.981607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:01.981607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:01.981607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:01.981607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:01.981607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:01.981607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:01.981607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:01.981607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:01.981607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:01.981607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:01.981607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:01.981607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:01.981607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:01.981607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:01.981607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:01.982369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:01.982369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:01.982369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:01.982369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:01.982369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:01.982369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:01.982369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:01.982369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:01.982369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:01.982369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:01.982369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:01.982369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:01.982369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 35/72] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:53:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:01.983023 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa6060-2e53-4f3b-99c6-2050620c8442 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:02.012196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212cb47b-3657-4452-8c91-1fb8aa080b7b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:02.051731 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c44586-4459-4609-a4f6-6fb97f31beb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:02.052457 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c44586-4459-4609-a4f6-6fb97f31beb7 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:02.052819 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c44586-4459-4609-a4f6-6fb97f31beb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:02.053007 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c44586-4459-4609-a4f6-6fb97f31beb7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:02.062596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:02.064827 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:02.069287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c44586-4459-4609-a4f6-6fb97f31beb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:02.071571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-78c44586-4459-4609-a4f6-6fb97f31beb7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:02.071571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:02.071571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:02.071571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:02.071571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:02.071571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:02.071571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:02.071571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:02.071571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:02.071571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:02.071571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:02.071571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:02.071571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:02.071571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:02.071571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:02.071571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:02.072576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:02.072576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:02.072576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:02.072576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:02.072576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:02.072576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:02.072576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:02.072576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:02.072576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:02.072576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:02.072576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:02.072576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:02.072576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:02.072576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:02.072576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:02.072576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:02.073336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:02.073336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:02.073336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:02.073336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:02.073336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:02.073336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:02.073336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:02.073336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:02.073336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:02.073336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:02.073336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:02.073336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:02.077551 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 38/73] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:53:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:02.078692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7424c7d-9099-4426-8d6b-9e527ddeb045 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:02.078970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7424c7d-9099-4426-8d6b-9e527ddeb045 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:02.079167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7424c7d-9099-4426-8d6b-9e527ddeb045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:02.086237 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa6060-2e53-4f3b-99c6-2050620c8442 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:02.096770 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-e7424c7d-9099-4426-8d6b-9e527ddeb045 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:02.097995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e7424c7d-9099-4426-8d6b-9e527ddeb045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:02.120447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a466213-4674-4d7f-b02d-b5821a7fa432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:02.121054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a466213-4674-4d7f-b02d-b5821a7fa432 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:02.121334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a466213-4674-4d7f-b02d-b5821a7fa432 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:02.121606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a466213-4674-4d7f-b02d-b5821a7fa432 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:02.122269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a466213-4674-4d7f-b02d-b5821a7fa432 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:02.123975 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a466213-4674-4d7f-b02d-b5821a7fa432 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:02.124259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a466213-4674-4d7f-b02d-b5821a7fa432 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:02.136314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 36/74] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:02.137235 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a466213-4674-4d7f-b02d-b5821a7fa432 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:02.168259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 39/75] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:02.169124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7424c7d-9099-4426-8d6b-9e527ddeb045 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:02.244189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a466213-4674-4d7f-b02d-b5821a7fa432 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:02.272684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7424c7d-9099-4426-8d6b-9e527ddeb045 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:02.284896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa4779-d516-4226-8912-188499c0f83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:02.285507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa4779-d516-4226-8912-188499c0f83c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:02.285788 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa4779-d516-4226-8912-188499c0f83c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:02.286315 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa4779-d516-4226-8912-188499c0f83c None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:02.287491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa4779-d516-4226-8912-188499c0f83c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:02.287806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa4779-d516-4226-8912-188499c0f83c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:02.289618 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa4779-d516-4226-8912-188499c0f83c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:02.299346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 37/76] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:02.300145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa4779-d516-4226-8912-188499c0f83c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:02.302888 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:02.306333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:02.312839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708adf4-9d28-4688-95e1-2495bef9e1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:02.315319 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708adf4-9d28-4688-95e1-2495bef9e1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:02.315319 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708adf4-9d28-4688-95e1-2495bef9e1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:02.315319 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708adf4-9d28-4688-95e1-2495bef9e1e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:02.315319 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708adf4-9d28-4688-95e1-2495bef9e1e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:02.315319 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708adf4-9d28-4688-95e1-2495bef9e1e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:02.316595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708adf4-9d28-4688-95e1-2495bef9e1e9 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:02.325208 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 40/77] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:53:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:02.326019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708adf4-9d28-4688-95e1-2495bef9e1e9 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:02.413371 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa4779-d516-4226-8912-188499c0f83c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:02.435858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708adf4-9d28-4688-95e1-2495bef9e1e9 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:02.449049 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f789ae-2f25-41ec-aad2-8ce327992d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:02.449677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f789ae-2f25-41ec-aad2-8ce327992d5f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:02.449949 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f789ae-2f25-41ec-aad2-8ce327992d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:02.450218 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f789ae-2f25-41ec-aad2-8ce327992d5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:02.460611 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f789ae-2f25-41ec-aad2-8ce327992d5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:02.466519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d0f789ae-2f25-41ec-aad2-8ce327992d5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:02.466519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:02.466519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:02.466519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:02.466519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:02.466519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:02.466519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:02.466519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:02.466519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:02.466519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:02.466519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:02.466519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:02.466519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:02.466519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:02.466519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:02.466519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:02.467421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:02.467421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:02.467421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:02.467421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:02.467421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:02.467421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:02.467421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:02.467421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:02.467421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:02.467421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:02.467421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:02.467421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:02.467421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:02.467421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:02.467421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:02.467421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:02.468180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:02.468180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:02.468180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:02.468180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:02.468180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:02.468180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:02.468180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:02.468180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:02.468180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:02.468180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:02.468180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:02.468180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:02.468180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 38/78] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:02.468815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f789ae-2f25-41ec-aad2-8ce327992d5f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:02.472974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4117f2-7fdb-4a6a-8a68-e48fdbd5edaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:02.473606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4117f2-7fdb-4a6a-8a68-e48fdbd5edaa None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:02.473885 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4117f2-7fdb-4a6a-8a68-e48fdbd5edaa None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:02.474158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4117f2-7fdb-4a6a-8a68-e48fdbd5edaa None admin] PATH_INFO: `/v3/users/ffb9e4ff6b76404b9d9afd02933c035e` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:02.488808 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4117f2-7fdb-4a6a-8a68-e48fdbd5edaa None admin] RBAC: Authorizing `identity:get_user(user_id=ffb9e4ff6b76404b9d9afd02933c035e)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:02.490274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4117f2-7fdb-4a6a-8a68-e48fdbd5edaa None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:02.491944 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 41/79] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:02 2022] GET /identity/v3/users/ffb9e4ff6b76404b9d9afd02933c035e => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:02.492714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4117f2-7fdb-4a6a-8a68-e48fdbd5edaa None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:02.529332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:02.541971 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:02.567754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f789ae-2f25-41ec-aad2-8ce327992d5f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:02.589502 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4117f2-7fdb-4a6a-8a68-e48fdbd5edaa None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:02.601546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d9b30-f114-447f-90fa-64fd4625166d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:02.602176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d9b30-f114-447f-90fa-64fd4625166d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:02.602459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d9b30-f114-447f-90fa-64fd4625166d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:02.602739 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d9b30-f114-447f-90fa-64fd4625166d None admin] PATH_INFO: `/v3/users/93747f497a1f4adcb4b8ddb03adfe01c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:02.604526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d9b30-f114-447f-90fa-64fd4625166d None admin] RBAC: Authorizing `identity:get_user(user_id=93747f497a1f4adcb4b8ddb03adfe01c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:02.605975 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d9b30-f114-447f-90fa-64fd4625166d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:02.607599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 39/80] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:02 2022] GET /identity/v3/users/93747f497a1f4adcb4b8ddb03adfe01c => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:02.608368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d9b30-f114-447f-90fa-64fd4625166d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:02.625376 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511759b9-bc11-4da4-b540-db1844ce5bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:02.625995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511759b9-bc11-4da4-b540-db1844ce5bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:02.626278 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511759b9-bc11-4da4-b540-db1844ce5bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:02.626550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511759b9-bc11-4da4-b540-db1844ce5bb3 None admin] PATH_INFO: `/v3/users/ffb9e4ff6b76404b9d9afd02933c035e` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:02.627718 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511759b9-bc11-4da4-b540-db1844ce5bb3 None admin] RBAC: Authorizing `identity:get_user(user_id=ffb9e4ff6b76404b9d9afd02933c035e)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:02.629167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511759b9-bc11-4da4-b540-db1844ce5bb3 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:02.630779 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 42/81] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:02 2022] GET /identity/v3/users/ffb9e4ff6b76404b9d9afd02933c035e => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:02.631564 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511759b9-bc11-4da4-b540-db1844ce5bb3 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:02.706554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d9b30-f114-447f-90fa-64fd4625166d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:02.730334 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511759b9-bc11-4da4-b540-db1844ce5bb3 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:02.734246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:02.739689 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8cd46-4a14-48f6-9aeb-8bf70bd0a477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:02.740328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8cd46-4a14-48f6-9aeb-8bf70bd0a477 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:02.740620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8cd46-4a14-48f6-9aeb-8bf70bd0a477 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:02.740898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8cd46-4a14-48f6-9aeb-8bf70bd0a477 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:02.742151 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8cd46-4a14-48f6-9aeb-8bf70bd0a477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:02.743640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8cd46-4a14-48f6-9aeb-8bf70bd0a477 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:02.745783 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 40/82] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:53:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:02.746551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8cd46-4a14-48f6-9aeb-8bf70bd0a477 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:02.748116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:02.764761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ea990-c195-4ceb-b7c7-f84f797c398e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:02.765377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ea990-c195-4ceb-b7c7-f84f797c398e None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:02.765663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ea990-c195-4ceb-b7c7-f84f797c398e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:02.765943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ea990-c195-4ceb-b7c7-f84f797c398e None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:02.766615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ea990-c195-4ceb-b7c7-f84f797c398e None admin] RBAC: Adding query filter params (name=member) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:02.766923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ea990-c195-4ceb-b7c7-f84f797c398e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:02.768341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ea990-c195-4ceb-b7c7-f84f797c398e None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:02.776345 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 43/83] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:02.777143 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ea990-c195-4ceb-b7c7-f84f797c398e None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:02.847025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8cd46-4a14-48f6-9aeb-8bf70bd0a477 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:02.879401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ea990-c195-4ceb-b7c7-f84f797c398e None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:02.880758 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21a23a-2518-4b22-8a3a-73179ba96ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:02.881358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21a23a-2518-4b22-8a3a-73179ba96ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:02.881638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21a23a-2518-4b22-8a3a-73179ba96ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:02.881905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21a23a-2518-4b22-8a3a-73179ba96ba8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:02.912359 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fedc1d-3b22-493c-b790-6f2e43bde30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:02.912968 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fedc1d-3b22-493c-b790-6f2e43bde30e None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:02.913253 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fedc1d-3b22-493c-b790-6f2e43bde30e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:02.913532 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fedc1d-3b22-493c-b790-6f2e43bde30e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:02.945334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:02.949218 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21a23a-2518-4b22-8a3a-73179ba96ba8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:02.950697 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21a23a-2518-4b22-8a3a-73179ba96ba8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:02.961886 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:02.981474 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fedc1d-3b22-493c-b790-6f2e43bde30e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:02.982966 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fedc1d-3b22-493c-b790-6f2e43bde30e None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:03.048624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 41/84] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:03.049412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21a23a-2518-4b22-8a3a-73179ba96ba8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:03.087353 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 44/85] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:03.088729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fedc1d-3b22-493c-b790-6f2e43bde30e None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:03.145882 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:03.149668 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21a23a-2518-4b22-8a3a-73179ba96ba8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:03.172566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:03.185947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569c5a8-d7cf-4bf8-b1d0-78a8ed470094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:03.186624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569c5a8-d7cf-4bf8-b1d0-78a8ed470094 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:03.187001 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569c5a8-d7cf-4bf8-b1d0-78a8ed470094 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:03.187279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569c5a8-d7cf-4bf8-b1d0-78a8ed470094 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:03.191116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fedc1d-3b22-493c-b790-6f2e43bde30e None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:03.225049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1a9a9-f46b-468b-b05e-e0df94872058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:03.225648 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1a9a9-f46b-468b-b05e-e0df94872058 None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:03.225928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1a9a9-f46b-468b-b05e-e0df94872058 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:03.226226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1a9a9-f46b-468b-b05e-e0df94872058 None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:03.226873 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1a9a9-f46b-468b-b05e-e0df94872058 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:03.228349 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1a9a9-f46b-468b-b05e-e0df94872058 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:03.255058 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569c5a8-d7cf-4bf8-b1d0-78a8ed470094 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:03.256537 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569c5a8-d7cf-4bf8-b1d0-78a8ed470094 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:03.298731 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 45/86] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:53:03 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:53:03.299604 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1a9a9-f46b-468b-b05e-e0df94872058 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:03.359812 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:03.365180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 42/87] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:03.366059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569c5a8-d7cf-4bf8-b1d0-78a8ed470094 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:03.440706 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1a9a9-f46b-468b-b05e-e0df94872058 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:03.477121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a718f7-c3b7-4092-b43b-71f084079ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:03.478027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a718f7-c3b7-4092-b43b-71f084079ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:03.478310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a718f7-c3b7-4092-b43b-71f084079ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:03.478639 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a718f7-c3b7-4092-b43b-71f084079ba3 None admin] PATH_INFO: `/v3/users/ffb9e4ff6b76404b9d9afd02933c035e` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:03.479085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569c5a8-d7cf-4bf8-b1d0-78a8ed470094 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:03.480556 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a718f7-c3b7-4092-b43b-71f084079ba3 None admin] RBAC: Authorizing `identity:get_user(user_id=ffb9e4ff6b76404b9d9afd02933c035e)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:03.482064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a718f7-c3b7-4092-b43b-71f084079ba3 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:03.483788 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 46/88] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:03 2022] GET /identity/v3/users/ffb9e4ff6b76404b9d9afd02933c035e => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:03.484650 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a718f7-c3b7-4092-b43b-71f084079ba3 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:03.490231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:03.520555 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b018f-01a9-42dc-a2f7-aa10aad325b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:03.521194 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b018f-01a9-42dc-a2f7-aa10aad325b3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:03.521470 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b018f-01a9-42dc-a2f7-aa10aad325b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:03.521742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b018f-01a9-42dc-a2f7-aa10aad325b3 None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:03.523487 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b018f-01a9-42dc-a2f7-aa10aad325b3 None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:03.525065 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b018f-01a9-42dc-a2f7-aa10aad325b3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:03.527491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 43/89] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:03 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:03.528176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b018f-01a9-42dc-a2f7-aa10aad325b3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:03.613859 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:03.615228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a718f7-c3b7-4092-b43b-71f084079ba3 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:03.659156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b66136-6092-43d1-864b-7f6622072cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:03.659801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b66136-6092-43d1-864b-7f6622072cbd None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:03.660038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b018f-01a9-42dc-a2f7-aa10aad325b3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:03.660138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b66136-6092-43d1-864b-7f6622072cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:03.660759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b66136-6092-43d1-864b-7f6622072cbd None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:03.662038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b66136-6092-43d1-864b-7f6622072cbd None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:03.664415 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b66136-6092-43d1-864b-7f6622072cbd None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:03.666189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 47/90] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:03 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:03.667058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b66136-6092-43d1-864b-7f6622072cbd None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:03.710036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fff63-f7c7-48af-9a49-76da3f96a2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:03.710592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fff63-f7c7-48af-9a49-76da3f96a2af None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:03.710873 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fff63-f7c7-48af-9a49-76da3f96a2af None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:03.711252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fff63-f7c7-48af-9a49-76da3f96a2af None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:03.712536 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fff63-f7c7-48af-9a49-76da3f96a2af None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:03.715225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fff63-f7c7-48af-9a49-76da3f96a2af None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:03.716967 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 44/91] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:03 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:03.717824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fff63-f7c7-48af-9a49-76da3f96a2af None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:03.751976 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:03.783869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b66136-6092-43d1-864b-7f6622072cbd None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:03.818448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb8a21-88f3-410a-978c-56b5c1b0f58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:03.819483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb8a21-88f3-410a-978c-56b5c1b0f58c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:03.819983 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb8a21-88f3-410a-978c-56b5c1b0f58c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:03.820445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb8a21-88f3-410a-978c-56b5c1b0f58c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:03.847490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fff63-f7c7-48af-9a49-76da3f96a2af None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:03.890507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323908b0-36b4-4a56-874b-5a713f7c0341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:03.891148 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323908b0-36b4-4a56-874b-5a713f7c0341 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:03.891433 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323908b0-36b4-4a56-874b-5a713f7c0341 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:03.891973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323908b0-36b4-4a56-874b-5a713f7c0341 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:03.892716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323908b0-36b4-4a56-874b-5a713f7c0341 None admin] RBAC: Adding query filter params (role.id=dd5cafa908a74cfd8e1a8baa44122242, scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45, user.id=ffb9e4ff6b76404b9d9afd02933c035e) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:03.892983 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb8a21-88f3-410a-978c-56b5c1b0f58c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:03.893082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323908b0-36b4-4a56-874b-5a713f7c0341 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:03.894534 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323908b0-36b4-4a56-874b-5a713f7c0341 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:03.894990 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb8a21-88f3-410a-978c-56b5c1b0f58c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:03.896865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:03.903830 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 45/92] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:03 2022] GET /identity/v3/role_assignments?user.id=ffb9e4ff6b76404b9d9afd02933c035e&scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45&role.id=dd5cafa908a74cfd8e1a8baa44122242 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:03.904987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323908b0-36b4-4a56-874b-5a713f7c0341 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:04.042712 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323908b0-36b4-4a56-874b-5a713f7c0341 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:04.055363 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:04.085831 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b31b8b-7745-48c3-86aa-8b2e3311cb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:04.086591 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b31b8b-7745-48c3-86aa-8b2e3311cb56 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:04.086962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b31b8b-7745-48c3-86aa-8b2e3311cb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:04.087319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b31b8b-7745-48c3-86aa-8b2e3311cb56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:04.088331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b31b8b-7745-48c3-86aa-8b2e3311cb56 None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45, user.id=93747f497a1f4adcb4b8ddb03adfe01c) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:04.088718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b31b8b-7745-48c3-86aa-8b2e3311cb56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:04.090343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b31b8b-7745-48c3-86aa-8b2e3311cb56 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:04.097193 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 46/93] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:03 2022] GET /identity/v3/role_assignments?user.id=93747f497a1f4adcb4b8ddb03adfe01c&scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:04.098528 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b31b8b-7745-48c3-86aa-8b2e3311cb56 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:04.110086 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 48/94] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:04.116037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb8a21-88f3-410a-978c-56b5c1b0f58c None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:04.200616 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:04.280490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b31b8b-7745-48c3-86aa-8b2e3311cb56 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:04.315804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb8a21-88f3-410a-978c-56b5c1b0f58c None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:04.383684 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55eaade-ba95-4048-bf49-b3bd241ef8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:04.384306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55eaade-ba95-4048-bf49-b3bd241ef8fd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:04.384600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55eaade-ba95-4048-bf49-b3bd241ef8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:04.384891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55eaade-ba95-4048-bf49-b3bd241ef8fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:04.385814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55eaade-ba95-4048-bf49-b3bd241ef8fd None admin] RBAC: Adding query filter params (role.id=d9b2237fd9884cc39771e75bf902e203, scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45, user.id=ffb9e4ff6b76404b9d9afd02933c035e) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:04.386129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55eaade-ba95-4048-bf49-b3bd241ef8fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:04.387607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55eaade-ba95-4048-bf49-b3bd241ef8fd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:04.388187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76065188-6c93-43d2-8230-1f025807b0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:04.389047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76065188-6c93-43d2-8230-1f025807b0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:04.389815 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76065188-6c93-43d2-8230-1f025807b0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:04.390311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76065188-6c93-43d2-8230-1f025807b0c8 None admin] PATH_INFO: `/v3/projects/bd37fdac1af54609810832817bd912d4` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:04.393143 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76065188-6c93-43d2-8230-1f025807b0c8 None admin] RBAC: Authorizing `identity:get_project(project_id=bd37fdac1af54609810832817bd912d4)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:04.394798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76065188-6c93-43d2-8230-1f025807b0c8 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:04.396978 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 49/95] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:04 2022] GET /identity/v3/projects/bd37fdac1af54609810832817bd912d4 => generated 312 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:04.399810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 47/96] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:04 2022] GET /identity/v3/role_assignments?user.id=ffb9e4ff6b76404b9d9afd02933c035e&scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45&role.id=d9b2237fd9884cc39771e75bf902e203 => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:04.403160 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76065188-6c93-43d2-8230-1f025807b0c8 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:04.464766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:04.537613 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76065188-6c93-43d2-8230-1f025807b0c8 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:04.579636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915c4ce-392f-47c0-bfdf-7927fa0b320e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:04.580573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915c4ce-392f-47c0-bfdf-7927fa0b320e None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:04.581045 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915c4ce-392f-47c0-bfdf-7927fa0b320e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:04.581503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915c4ce-392f-47c0-bfdf-7927fa0b320e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:04.582423 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915c4ce-392f-47c0-bfdf-7927fa0b320e None admin] RBAC: Adding query filter params (role.id=dd5cafa908a74cfd8e1a8baa44122242, scope.project.id=bd37fdac1af54609810832817bd912d4, user.id=ffb9e4ff6b76404b9d9afd02933c035e) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:04.582959 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915c4ce-392f-47c0-bfdf-7927fa0b320e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:04.584668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915c4ce-392f-47c0-bfdf-7927fa0b320e None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:04.595042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 50/97] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:04 2022] GET /identity/v3/role_assignments?user.id=ffb9e4ff6b76404b9d9afd02933c035e&scope.project.id=bd37fdac1af54609810832817bd912d4&role.id=dd5cafa908a74cfd8e1a8baa44122242 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:06.660216 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1e2d1-c02e-48d5-b44e-74873a17ce0d None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:06.660721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1e2d1-c02e-48d5-b44e-74873a17ce0d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:06.660822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1e2d1-c02e-48d5-b44e-74873a17ce0d None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:06.662414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 48/98] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:06.668636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752e387-6eb4-496e-9ebb-e5df22ceb0a5 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:06.669095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752e387-6eb4-496e-9ebb-e5df22ceb0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:06.669196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752e387-6eb4-496e-9ebb-e5df22ceb0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:06.707972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-5752e387-6eb4-496e-9ebb-e5df22ceb0a5 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:06.709201 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5752e387-6eb4-496e-9ebb-e5df22ceb0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:06.794287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6dbfd-237d-4571-b8d7-9cce29b9d8ba None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:06.794636 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6dbfd-237d-4571-b8d7-9cce29b9d8ba None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:06.794842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6dbfd-237d-4571-b8d7-9cce29b9d8ba None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:06.796395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 49/99] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:06.802035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9684a-0167-4889-9f22-9080ff4cdf26 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:06.802320 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9684a-0167-4889-9f22-9080ff4cdf26 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:06.802598 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9684a-0167-4889-9f22-9080ff4cdf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:06.833314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 51/100] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:06.839586 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-52d9684a-0167-4889-9f22-9080ff4cdf26 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:06.840806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-52d9684a-0167-4889-9f22-9080ff4cdf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:06.914053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad7e29-719f-478e-af43-8a0eaa63a624 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:06.914336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad7e29-719f-478e-af43-8a0eaa63a624 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:06.914543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad7e29-719f-478e-af43-8a0eaa63a624 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:06.916536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 52/101] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:53:06.921522 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5633f-f71f-4122-b078-109e97a9909c None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:06.921809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5633f-f71f-4122-b078-109e97a9909c None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:06.922095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5633f-f71f-4122-b078-109e97a9909c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:06.946085 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-99b5633f-f71f-4122-b078-109e97a9909c None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:06.947409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-99b5633f-f71f-4122-b078-109e97a9909c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:06.949645 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 50/102] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:06.996027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:07.052360 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 53/103] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:07.248165 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9684a-0167-4889-9f22-9080ff4cdf26 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:07.265151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767086f-7a7b-41e9-b1eb-e3ed079a245d None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:07.265443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767086f-7a7b-41e9-b1eb-e3ed079a245d None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:07.265707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767086f-7a7b-41e9-b1eb-e3ed079a245d None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:07.267316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 54/104] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:07.272369 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f2f2c-47e1-4a87-b5be-8a1a3345f56b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:07.272650 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f2f2c-47e1-4a87-b5be-8a1a3345f56b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:07.272922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f2f2c-47e1-4a87-b5be-8a1a3345f56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:07.297458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-f69f2f2c-47e1-4a87-b5be-8a1a3345f56b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:07.298649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f69f2f2c-47e1-4a87-b5be-8a1a3345f56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:07.325667 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:07.386360 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9684a-0167-4889-9f22-9080ff4cdf26 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:07.441970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 55/105] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:07.444083 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171ebba-e9fb-4280-92de-1e1f4bc91cd9 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:07.444367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171ebba-e9fb-4280-92de-1e1f4bc91cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:07.444642 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171ebba-e9fb-4280-92de-1e1f4bc91cd9 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:07.445913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 56/106] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:07.450502 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171ebba-e9fb-4280-92de-1e1f4bc91cd9 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:07.459474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254a552-255d-47cd-bcfe-f55e65c7ec4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:07.460118 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254a552-255d-47cd-bcfe-f55e65c7ec4b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:07.460398 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254a552-255d-47cd-bcfe-f55e65c7ec4b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:07.460671 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254a552-255d-47cd-bcfe-f55e65c7ec4b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:07.474764 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254a552-255d-47cd-bcfe-f55e65c7ec4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:07.481710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d254a552-255d-47cd-bcfe-f55e65c7ec4b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:07.481710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:07.481710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:07.481710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:07.481710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:07.481710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:07.481710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:07.481710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:07.481710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:07.481710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:07.481710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:07.481710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:07.481710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:07.481710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:07.481710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:07.481710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:07.482766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:07.482766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:07.482766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:07.482766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:07.482766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:07.482766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:07.482766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:07.482766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:07.482766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:07.482766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:07.482766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:07.482766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:07.482766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:07.482766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:07.482766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:07.482766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:07.483567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:07.483567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:07.483567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:07.483567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:07.483567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:07.483567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:07.483567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:07.483567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:07.483567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:07.483567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:07.483567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:07.483567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:07.484335 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 51/107] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:07.485328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c359338-7041-408c-923d-d98276206442 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:07.485616 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c359338-7041-408c-923d-d98276206442 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:07.485888 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c359338-7041-408c-923d-d98276206442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:07.506435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8c359338-7041-408c-923d-d98276206442 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:07.507656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8c359338-7041-408c-923d-d98276206442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:07.562708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:07.572154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171ebba-e9fb-4280-92de-1e1f4bc91cd9 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:07.591204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 52/108] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:07.592476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c359338-7041-408c-923d-d98276206442 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:07.637359 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c47f2-565a-4bf8-8b4b-8a63a971f6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:07.639141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c47f2-565a-4bf8-8b4b-8a63a971f6e1 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:07.639813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c47f2-565a-4bf8-8b4b-8a63a971f6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:07.641172 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c47f2-565a-4bf8-8b4b-8a63a971f6e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:07.643443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c47f2-565a-4bf8-8b4b-8a63a971f6e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:07.645891 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c47f2-565a-4bf8-8b4b-8a63a971f6e1 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:07.648577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 57/109] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:53:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:07.651691 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c47f2-565a-4bf8-8b4b-8a63a971f6e1 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:07.686311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:07.775293 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c359338-7041-408c-923d-d98276206442 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:07.805936 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c47f2-565a-4bf8-8b4b-8a63a971f6e1 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:07.813578 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be3a42-096b-4be8-9106-42d6ea7ceb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:07.814187 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be3a42-096b-4be8-9106-42d6ea7ceb92 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:07.814464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be3a42-096b-4be8-9106-42d6ea7ceb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:07.814731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be3a42-096b-4be8-9106-42d6ea7ceb92 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:07.815471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be3a42-096b-4be8-9106-42d6ea7ceb92 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:07.815774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be3a42-096b-4be8-9106-42d6ea7ceb92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:07.817286 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be3a42-096b-4be8-9106-42d6ea7ceb92 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:07.826886 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 53/110] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:07.832305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be3a42-096b-4be8-9106-42d6ea7ceb92 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:07.847122 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6040d0-5c76-48c9-b095-ad55ad23da5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:07.848089 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6040d0-5c76-48c9-b095-ad55ad23da5f None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:07.848652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6040d0-5c76-48c9-b095-ad55ad23da5f None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:07.849120 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6040d0-5c76-48c9-b095-ad55ad23da5f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:07.860052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6040d0-5c76-48c9-b095-ad55ad23da5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:07.865456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-ef6040d0-5c76-48c9-b095-ad55ad23da5f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:07.865456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:07.865456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:07.865456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:07.865456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:07.865456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:07.865456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:07.865456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:07.865456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:07.865456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:07.865456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:07.865456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:07.865456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:07.865456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:07.865456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:07.865456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:07.866332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:07.866332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:07.866332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:07.866332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:07.866332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:07.866332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:07.866332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:07.866332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:07.866332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:07.866332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:07.866332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:07.866332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:07.866332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:07.866332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:07.866332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:07.866332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:07.867109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:07.867109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:07.867109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:07.867109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:07.867109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:07.867109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:07.867109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:07.867109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:07.867109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:07.867109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:07.867109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:07.867109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:07.869971 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 58/111] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:07.871095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6040d0-5c76-48c9-b095-ad55ad23da5f None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:07.898652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:07.948205 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be3a42-096b-4be8-9106-42d6ea7ceb92 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:07.957687 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:07.976628 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6040d0-5c76-48c9-b095-ad55ad23da5f None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:07.992440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746cc0c-12fe-4c62-b8a0-caf79a669da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:07.993237 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746cc0c-12fe-4c62-b8a0-caf79a669da1 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:07.993516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746cc0c-12fe-4c62-b8a0-caf79a669da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:07.993789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746cc0c-12fe-4c62-b8a0-caf79a669da1 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:07.995228 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746cc0c-12fe-4c62-b8a0-caf79a669da1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:07.997107 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746cc0c-12fe-4c62-b8a0-caf79a669da1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:08.017568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96113d62-5490-4bb6-b6bb-a2bbc8fbc4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:08.018112 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96113d62-5490-4bb6-b6bb-a2bbc8fbc4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:08.018417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96113d62-5490-4bb6-b6bb-a2bbc8fbc4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:08.018695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96113d62-5490-4bb6-b6bb-a2bbc8fbc4d5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:08.029216 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96113d62-5490-4bb6-b6bb-a2bbc8fbc4d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:08.033973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-96113d62-5490-4bb6-b6bb-a2bbc8fbc4d5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:08.033973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:08.033973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:08.033973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:08.033973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:08.033973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:08.033973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:08.033973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:08.033973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:08.033973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:08.033973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:08.033973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:08.033973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:08.033973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:08.033973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:08.033973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:08.039059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:08.039059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:08.039059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:08.039059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:08.039059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:08.039059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:08.039059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:08.039059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:08.039059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:08.039059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:08.039059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:08.039059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:08.039059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:08.039059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:08.039059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:08.039059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:08.040412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:08.040412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:08.040412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:08.040412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:08.040412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:08.040412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:08.040412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:08.040412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:08.040412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:08.040412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:08.040412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:08.040412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:08.040412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 59/112] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:53:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:08.040412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96113d62-5490-4bb6-b6bb-a2bbc8fbc4d5 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:08.072188 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 54/113] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Oct 6 07:53:07 2022] POST /identity/v3/projects => generated 302 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:53:08.073199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746cc0c-12fe-4c62-b8a0-caf79a669da1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:08.139770 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:08.165389 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96113d62-5490-4bb6-b6bb-a2bbc8fbc4d5 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:08.203508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746cc0c-12fe-4c62-b8a0-caf79a669da1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:08.203861 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46aee65-6cfe-4300-b227-b5cb1bac33d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:08.204453 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46aee65-6cfe-4300-b227-b5cb1bac33d9 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:08.204735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46aee65-6cfe-4300-b227-b5cb1bac33d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:08.205011 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46aee65-6cfe-4300-b227-b5cb1bac33d9 None admin] PATH_INFO: `/v3/users/ffb9e4ff6b76404b9d9afd02933c035e` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:08.206381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46aee65-6cfe-4300-b227-b5cb1bac33d9 None admin] RBAC: Authorizing `identity:get_user(user_id=ffb9e4ff6b76404b9d9afd02933c035e)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:08.207956 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46aee65-6cfe-4300-b227-b5cb1bac33d9 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:08.214466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 60/114] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:08 2022] GET /identity/v3/users/ffb9e4ff6b76404b9d9afd02933c035e => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:08.214466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46aee65-6cfe-4300-b227-b5cb1bac33d9 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:08.249532 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f514fb4-7446-458d-962f-1a5b774a9955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:08.250997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f514fb4-7446-458d-962f-1a5b774a9955 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:08.251426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f514fb4-7446-458d-962f-1a5b774a9955 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:08.251705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f514fb4-7446-458d-962f-1a5b774a9955 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:08.252383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f514fb4-7446-458d-962f-1a5b774a9955 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:08.252685 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f514fb4-7446-458d-962f-1a5b774a9955 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:08.254062 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f514fb4-7446-458d-962f-1a5b774a9955 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:08.264655 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 55/115] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:08.265519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f514fb4-7446-458d-962f-1a5b774a9955 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:08.293574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:08.328763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46aee65-6cfe-4300-b227-b5cb1bac33d9 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:08.364432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de93ad-10e4-4615-b0b4-819d25aa1a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:08.365136 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de93ad-10e4-4615-b0b4-819d25aa1a61 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:08.365358 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de93ad-10e4-4615-b0b4-819d25aa1a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:08.365633 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de93ad-10e4-4615-b0b4-819d25aa1a61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:08.376505 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de93ad-10e4-4615-b0b4-819d25aa1a61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:08.377045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f514fb4-7446-458d-962f-1a5b774a9955 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:08.381626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-a5de93ad-10e4-4615-b0b4-819d25aa1a61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:08.381626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:08.381626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:08.381626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:08.381626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:08.381626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:08.381626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:08.381626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:08.381626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:08.381626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:08.381626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:08.381626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:08.381626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:08.381626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:08.381626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:08.381626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:08.383286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:08.383286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:08.383286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:08.383286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:08.383286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:08.383286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:08.383286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:08.383286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:08.383286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:08.383286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:08.383286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:08.383286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:08.383286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:08.383286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:08.383286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:08.383286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:08.384554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:08.384554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:08.384554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:08.384554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:08.384554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:08.384554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:08.384554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:08.384554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:08.384554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:08.384554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:08.384554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:08.384554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:08.384554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 61/116] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:53:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:08.384554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8481b1-87e1-4f06-b0d9-b34540853630 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:08.385380 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8481b1-87e1-4f06-b0d9-b34540853630 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:08.385380 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8481b1-87e1-4f06-b0d9-b34540853630 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:08.386464 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 62/117] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:08.387516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8481b1-87e1-4f06-b0d9-b34540853630 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:08.403121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:08.419249 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d9530-1773-4586-84a2-89bf80f20ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:08.419923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d9530-1773-4586-84a2-89bf80f20ebd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:08.420192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d9530-1773-4586-84a2-89bf80f20ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:08.420464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d9530-1773-4586-84a2-89bf80f20ebd None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:08.421127 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d9530-1773-4586-84a2-89bf80f20ebd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:08.421428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d9530-1773-4586-84a2-89bf80f20ebd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:08.422809 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d9530-1773-4586-84a2-89bf80f20ebd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:08.430884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 56/118] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:53:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:08.432036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d9530-1773-4586-84a2-89bf80f20ebd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:08.500827 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8481b1-87e1-4f06-b0d9-b34540853630 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:08.520459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:08.538712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90c866-9731-43f4-bcee-878039f833d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:08.539394 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90c866-9731-43f4-bcee-878039f833d5 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:08.539678 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90c866-9731-43f4-bcee-878039f833d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:08.539960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90c866-9731-43f4-bcee-878039f833d5 None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:08.541843 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90c866-9731-43f4-bcee-878039f833d5 None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:08.542967 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d9530-1773-4586-84a2-89bf80f20ebd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:08.543262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90c866-9731-43f4-bcee-878039f833d5 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:08.545012 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 63/119] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:08 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:08.545733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90c866-9731-43f4-bcee-878039f833d5 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:08.586080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d16d8-f5c1-45e8-a931-8f861e29e804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:08.587222 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d16d8-f5c1-45e8-a931-8f861e29e804 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:08.588265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d16d8-f5c1-45e8-a931-8f861e29e804 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:08.589230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d16d8-f5c1-45e8-a931-8f861e29e804 None admin] PATH_INFO: `/v3/users/b351dfc51ec94ee29ccd78418e90d247` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:08.609783 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d16d8-f5c1-45e8-a931-8f861e29e804 None admin] RBAC: Authorizing `identity:get_user(user_id=b351dfc51ec94ee29ccd78418e90d247)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:08.611595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d16d8-f5c1-45e8-a931-8f861e29e804 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:08.613499 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 57/120] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:08 2022] GET /identity/v3/users/b351dfc51ec94ee29ccd78418e90d247 => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:08.614913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fc1c1-278b-4e26-a766-09b768fc5691 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:08.615207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fc1c1-278b-4e26-a766-09b768fc5691 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:08.615479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fc1c1-278b-4e26-a766-09b768fc5691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:08.628172 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:08.634202 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fa6fc1c1-278b-4e26-a766-09b768fc5691 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:08.635704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fa6fc1c1-278b-4e26-a766-09b768fc5691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:08.651219 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90c866-9731-43f4-bcee-878039f833d5 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:08.687322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fdbad-18ea-4217-95d8-10cbdf7d6f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:08.687936 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fdbad-18ea-4217-95d8-10cbdf7d6f62 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:08.688217 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fdbad-18ea-4217-95d8-10cbdf7d6f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:08.688492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fdbad-18ea-4217-95d8-10cbdf7d6f62 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:08.689162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fdbad-18ea-4217-95d8-10cbdf7d6f62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:08.689468 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fdbad-18ea-4217-95d8-10cbdf7d6f62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:08.690869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fdbad-18ea-4217-95d8-10cbdf7d6f62 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:08.699634 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 64/121] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:08.700441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fdbad-18ea-4217-95d8-10cbdf7d6f62 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:08.712332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 58/122] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:08.714198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fc1c1-278b-4e26-a766-09b768fc5691 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:08.806188 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:08.824666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fdbad-18ea-4217-95d8-10cbdf7d6f62 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:08.841478 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fc1c1-278b-4e26-a766-09b768fc5691 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:08.862457 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b84ee0-9a78-46ba-a6d4-c0e35136e6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:08.863270 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b84ee0-9a78-46ba-a6d4-c0e35136e6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:08.863698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b84ee0-9a78-46ba-a6d4-c0e35136e6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:08.864050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b84ee0-9a78-46ba-a6d4-c0e35136e6e3 None admin] PATH_INFO: `/v3/users/ffb9e4ff6b76404b9d9afd02933c035e` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:08.865343 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b84ee0-9a78-46ba-a6d4-c0e35136e6e3 None admin] RBAC: Authorizing `identity:get_user(user_id=ffb9e4ff6b76404b9d9afd02933c035e)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:08.866970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b84ee0-9a78-46ba-a6d4-c0e35136e6e3 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:08.869053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 65/123] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:08 2022] GET /identity/v3/users/ffb9e4ff6b76404b9d9afd02933c035e => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:08.869992 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b84ee0-9a78-46ba-a6d4-c0e35136e6e3 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:08.880474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14bf9b2-2629-45d9-8a06-c3da79f26843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:08.881400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bf9b2-2629-45d9-8a06-c3da79f26843 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:08.881848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bf9b2-2629-45d9-8a06-c3da79f26843 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:08.882283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bf9b2-2629-45d9-8a06-c3da79f26843 None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45/users/ffb9e4ff6b76404b9d9afd02933c035e/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:08.882697 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:08.885918 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14bf9b2-2629-45d9-8a06-c3da79f26843 None admin] RBAC: Authorizing `identity:create_grant(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45, user_id=ffb9e4ff6b76404b9d9afd02933c035e, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:08.887592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14bf9b2-2629-45d9-8a06-c3da79f26843 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:08.976307 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 59/124] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:53:08 2022] PUT /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45/users/ffb9e4ff6b76404b9d9afd02933c035e/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:53:08.976307 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14bf9b2-2629-45d9-8a06-c3da79f26843 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:08.988167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b84ee0-9a78-46ba-a6d4-c0e35136e6e3 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:09.042240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897dd4a-c0bf-48a5-8aa6-f7fffb0498d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:09.043921 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897dd4a-c0bf-48a5-8aa6-f7fffb0498d8 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:09.044639 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897dd4a-c0bf-48a5-8aa6-f7fffb0498d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:09.045134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897dd4a-c0bf-48a5-8aa6-f7fffb0498d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:09.117040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14bf9b2-2629-45d9-8a06-c3da79f26843 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:09.150777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897dd4a-c0bf-48a5-8aa6-f7fffb0498d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:09.152774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897dd4a-c0bf-48a5-8aa6-f7fffb0498d8 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:09.160486 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:09.167043 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:09.170194 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba61bc8-6cf9-4416-a3fb-0940e11a50cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:09.170857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba61bc8-6cf9-4416-a3fb-0940e11a50cb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:09.174582 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba61bc8-6cf9-4416-a3fb-0940e11a50cb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:09.174582 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba61bc8-6cf9-4416-a3fb-0940e11a50cb None admin] PATH_INFO: `/v3/users/93747f497a1f4adcb4b8ddb03adfe01c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:09.174582 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba61bc8-6cf9-4416-a3fb-0940e11a50cb None admin] RBAC: Authorizing `identity:get_user(user_id=93747f497a1f4adcb4b8ddb03adfe01c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:09.174582 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba61bc8-6cf9-4416-a3fb-0940e11a50cb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:09.176322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 60/125] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:08 2022] GET /identity/v3/users/93747f497a1f4adcb4b8ddb03adfe01c => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:09.177058 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba61bc8-6cf9-4416-a3fb-0940e11a50cb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:09.289868 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 66/126] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:09.295141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897dd4a-c0bf-48a5-8aa6-f7fffb0498d8 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:09.298200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba61bc8-6cf9-4416-a3fb-0940e11a50cb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:09.335202 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6e1e1-8bf8-4a55-a454-2db48d39da10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:09.335838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6e1e1-8bf8-4a55-a454-2db48d39da10 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:09.336117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6e1e1-8bf8-4a55-a454-2db48d39da10 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:09.336390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6e1e1-8bf8-4a55-a454-2db48d39da10 None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:09.338454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6e1e1-8bf8-4a55-a454-2db48d39da10 None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:09.340143 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6e1e1-8bf8-4a55-a454-2db48d39da10 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:09.341946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 61/127] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:09 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:09.342825 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6e1e1-8bf8-4a55-a454-2db48d39da10 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:09.401234 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897dd4a-c0bf-48a5-8aa6-f7fffb0498d8 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:09.411200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:09.417133 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:09.437701 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f425f84-1b03-4a6f-a827-35c7a6347bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:09.438694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f425f84-1b03-4a6f-a827-35c7a6347bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:09.439202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f425f84-1b03-4a6f-a827-35c7a6347bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:09.439665 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f425f84-1b03-4a6f-a827-35c7a6347bd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:09.451125 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f425f84-1b03-4a6f-a827-35c7a6347bd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:09.453691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6e1e1-8bf8-4a55-a454-2db48d39da10 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:09.456624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-9f425f84-1b03-4a6f-a827-35c7a6347bd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:09.456624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:09.456624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:09.456624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:09.456624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:09.456624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:09.456624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:09.456624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:09.456624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:09.456624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:09.456624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:09.456624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:09.456624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:09.456624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:09.456624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:09.456624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:09.457480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:09.457480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:09.457480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:09.457480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:09.457480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:09.457480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:09.457480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:09.457480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:09.457480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:09.457480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:09.457480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:09.457480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:09.457480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:09.457480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:09.457480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:09.457480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:09.458289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:09.458289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:09.458289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:09.458289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:09.458289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:09.458289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:09.458289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:09.458289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:09.458289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:09.458289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:09.458289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:09.458289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:09.460910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 67/128] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:09.462079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f425f84-1b03-4a6f-a827-35c7a6347bd6 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:09.492706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb483a6b-0b14-4b82-b6c7-cea27cdb3043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:09.493369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb483a6b-0b14-4b82-b6c7-cea27cdb3043 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:09.493650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb483a6b-0b14-4b82-b6c7-cea27cdb3043 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:09.494637 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb483a6b-0b14-4b82-b6c7-cea27cdb3043 None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:09.495737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb483a6b-0b14-4b82-b6c7-cea27cdb3043 None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:09.497443 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb483a6b-0b14-4b82-b6c7-cea27cdb3043 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:09.499459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 62/129] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:09 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:09.500244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb483a6b-0b14-4b82-b6c7-cea27cdb3043 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:09.568413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f425f84-1b03-4a6f-a827-35c7a6347bd6 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:09.603487 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa05bb4-9e46-40b7-be10-0ec527466eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:09.604513 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa05bb4-9e46-40b7-be10-0ec527466eac None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:09.605007 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa05bb4-9e46-40b7-be10-0ec527466eac None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:09.605469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa05bb4-9e46-40b7-be10-0ec527466eac None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:09.606816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa05bb4-9e46-40b7-be10-0ec527466eac None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:09.608513 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa05bb4-9e46-40b7-be10-0ec527466eac None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:09.615083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb483a6b-0b14-4b82-b6c7-cea27cdb3043 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:09.615328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 68/130] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:09 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:09.616317 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa05bb4-9e46-40b7-be10-0ec527466eac None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:09.649154 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb48a77-22ce-4b53-b074-b69173142250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:09.649777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb48a77-22ce-4b53-b074-b69173142250 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:09.650688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb48a77-22ce-4b53-b074-b69173142250 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:09.651032 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb48a77-22ce-4b53-b074-b69173142250 None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45/users/ffb9e4ff6b76404b9d9afd02933c035e/roles/d9b2237fd9884cc39771e75bf902e203` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:09.653513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb48a77-22ce-4b53-b074-b69173142250 None admin] RBAC: Authorizing `identity:create_grant(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45, user_id=ffb9e4ff6b76404b9d9afd02933c035e, role_id=d9b2237fd9884cc39771e75bf902e203)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:09.654911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb48a77-22ce-4b53-b074-b69173142250 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:09.658240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:09.659213 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:09.723978 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa05bb4-9e46-40b7-be10-0ec527466eac None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:09.734959 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 63/131] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:53:09 2022] PUT /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45/users/ffb9e4ff6b76404b9d9afd02933c035e/roles/d9b2237fd9884cc39771e75bf902e203 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:53:09.737050 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb48a77-22ce-4b53-b074-b69173142250 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:09.788350 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a6f05-90db-4457-b399-9406b8570a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:09.789391 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a6f05-90db-4457-b399-9406b8570a01 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:09.789868 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a6f05-90db-4457-b399-9406b8570a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:09.790334 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a6f05-90db-4457-b399-9406b8570a01 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:09.791723 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a6f05-90db-4457-b399-9406b8570a01 None admin] RBAC: Adding query filter params (name=member) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:09.793629 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a6f05-90db-4457-b399-9406b8570a01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:09.796296 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a6f05-90db-4457-b399-9406b8570a01 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:09.814247 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 69/132] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:09.815995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a6f05-90db-4457-b399-9406b8570a01 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:09.938015 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:09.943093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a6f05-90db-4457-b399-9406b8570a01 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:09.958523 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb48a77-22ce-4b53-b074-b69173142250 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:09.981297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d45d7-8164-4945-ae91-d1255ab6a1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:09.982295 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d45d7-8164-4945-ae91-d1255ab6a1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:09.982797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d45d7-8164-4945-ae91-d1255ab6a1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:09.983413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d45d7-8164-4945-ae91-d1255ab6a1e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:09.984412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d45d7-8164-4945-ae91-d1255ab6a1e8 None admin] RBAC: Adding query filter params (role.id=74955b19ba90422cbe29fa461c3f287f, scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45, user.id=b351dfc51ec94ee29ccd78418e90d247) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:09.985112 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d45d7-8164-4945-ae91-d1255ab6a1e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:09.987011 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d45d7-8164-4945-ae91-d1255ab6a1e8 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:09.994541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 70/133] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:09 2022] GET /identity/v3/role_assignments?user.id=b351dfc51ec94ee29ccd78418e90d247&scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45&role.id=74955b19ba90422cbe29fa461c3f287f => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:09.995716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d45d7-8164-4945-ae91-d1255ab6a1e8 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:10.064621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a858c2-c7a3-4f68-9619-2ab7425f09f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:10.065308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a858c2-c7a3-4f68-9619-2ab7425f09f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:10.065592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a858c2-c7a3-4f68-9619-2ab7425f09f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:10.065871 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a858c2-c7a3-4f68-9619-2ab7425f09f4 None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45/users/93747f497a1f4adcb4b8ddb03adfe01c/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:10.073173 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a858c2-c7a3-4f68-9619-2ab7425f09f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45, user_id=93747f497a1f4adcb4b8ddb03adfe01c, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:10.074854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a858c2-c7a3-4f68-9619-2ab7425f09f4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:10.116588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d45d7-8164-4945-ae91-d1255ab6a1e8 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:10.168128 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a8558-d971-4039-89df-5bd3e39b015b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:10.168890 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a8558-d971-4039-89df-5bd3e39b015b None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:10.169269 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a8558-d971-4039-89df-5bd3e39b015b None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:10.169696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a8558-d971-4039-89df-5bd3e39b015b None admin] PATH_INFO: `/v3/users/ffb9e4ff6b76404b9d9afd02933c035e` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:10.171841 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a8558-d971-4039-89df-5bd3e39b015b None admin] RBAC: Authorizing `identity:get_user(user_id=ffb9e4ff6b76404b9d9afd02933c035e)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:10.173415 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a8558-d971-4039-89df-5bd3e39b015b None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:10.175615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 71/134] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:09 2022] GET /identity/v3/users/ffb9e4ff6b76404b9d9afd02933c035e => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:10.182054 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a8558-d971-4039-89df-5bd3e39b015b None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:10.194958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 64/135] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:53:09 2022] PUT /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45/users/93747f497a1f4adcb4b8ddb03adfe01c/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:53:10.242121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:10.327647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a8558-d971-4039-89df-5bd3e39b015b None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:10.375424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0e278-d07d-43bf-b7a7-b5d0d816cdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:10.376520 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0e278-d07d-43bf-b7a7-b5d0d816cdc6 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:10.377086 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0e278-d07d-43bf-b7a7-b5d0d816cdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:10.377559 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0e278-d07d-43bf-b7a7-b5d0d816cdc6 None admin] PATH_INFO: `/v3/projects/bd37fdac1af54609810832817bd912d4` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:10.379058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0e278-d07d-43bf-b7a7-b5d0d816cdc6 None admin] RBAC: Authorizing `identity:get_project(project_id=bd37fdac1af54609810832817bd912d4)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:10.380793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0e278-d07d-43bf-b7a7-b5d0d816cdc6 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:10.382803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 72/136] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:10 2022] GET /identity/v3/projects/bd37fdac1af54609810832817bd912d4 => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:10.389271 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a858c2-c7a3-4f68-9619-2ab7425f09f4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:10.418067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56a6e5-d50c-4a10-9143-e7fbd9596daf None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:10.418700 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56a6e5-d50c-4a10-9143-e7fbd9596daf None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:10.419217 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56a6e5-d50c-4a10-9143-e7fbd9596daf None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:10.420766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 73/137] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:10.434375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecdde6-73de-46d2-b5cf-a7c7a0a41171 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:10.435048 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecdde6-73de-46d2-b5cf-a7c7a0a41171 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:10.435785 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecdde6-73de-46d2-b5cf-a7c7a0a41171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:10.467703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-37ecdde6-73de-46d2-b5cf-a7c7a0a41171 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:10.469389 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-37ecdde6-73de-46d2-b5cf-a7c7a0a41171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:10.483465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:10.567414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a858c2-c7a3-4f68-9619-2ab7425f09f4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:10.608216 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 74/138] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:10.621979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836662e7-a26d-41ae-95ae-5de3c7491f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:10.622960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836662e7-a26d-41ae-95ae-5de3c7491f0d None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:10.623698 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836662e7-a26d-41ae-95ae-5de3c7491f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:10.624194 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836662e7-a26d-41ae-95ae-5de3c7491f0d None admin] PATH_INFO: `/v3/projects/bd37fdac1af54609810832817bd912d4/users/ffb9e4ff6b76404b9d9afd02933c035e/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:10.626365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836662e7-a26d-41ae-95ae-5de3c7491f0d None admin] RBAC: Authorizing `identity:create_grant(project_id=bd37fdac1af54609810832817bd912d4, user_id=ffb9e4ff6b76404b9d9afd02933c035e, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:10.628067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836662e7-a26d-41ae-95ae-5de3c7491f0d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:10.705927 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 65/139] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:53:10 2022] PUT /identity/v3/projects/bd37fdac1af54609810832817bd912d4/users/ffb9e4ff6b76404b9d9afd02933c035e/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:53:10.982703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecdde6-73de-46d2-b5cf-a7c7a0a41171 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:11.015322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:11.105937 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecdde6-73de-46d2-b5cf-a7c7a0a41171 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:11.159876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158c309-3509-4bce-ab06-8c9b4e3f0ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:11.160624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158c309-3509-4bce-ab06-8c9b4e3f0ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:11.160910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158c309-3509-4bce-ab06-8c9b4e3f0ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:11.161186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158c309-3509-4bce-ab06-8c9b4e3f0ea4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:11.162464 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158c309-3509-4bce-ab06-8c9b4e3f0ea4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:11.164091 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158c309-3509-4bce-ab06-8c9b4e3f0ea4 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:11.166436 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 75/140] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:53:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:11.172367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836662e7-a26d-41ae-95ae-5de3c7491f0d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:11.282391 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836662e7-a26d-41ae-95ae-5de3c7491f0d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:11.300013 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:11.322140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993788d3-9bd7-4019-bced-8fd7d59e2952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:11.322847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993788d3-9bd7-4019-bced-8fd7d59e2952 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:11.323152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993788d3-9bd7-4019-bced-8fd7d59e2952 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:11.323429 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993788d3-9bd7-4019-bced-8fd7d59e2952 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:11.324108 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993788d3-9bd7-4019-bced-8fd7d59e2952 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:11.325579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993788d3-9bd7-4019-bced-8fd7d59e2952 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:11.449238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 66/141] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:53:11 2022] POST /identity/v3/users => generated 283 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:53:11.955946 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f192f-db9e-4335-9f17-5f69bf879564 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:11.956232 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f192f-db9e-4335-9f17-5f69bf879564 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:11.956587 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f192f-db9e-4335-9f17-5f69bf879564 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:11.958081 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 76/142] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:11.963571 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302d9c5-34ff-4440-a75f-16b8d1b023b3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:11.963869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302d9c5-34ff-4440-a75f-16b8d1b023b3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:11.964148 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302d9c5-34ff-4440-a75f-16b8d1b023b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:11.983889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b302d9c5-34ff-4440-a75f-16b8d1b023b3 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:11.985083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b302d9c5-34ff-4440-a75f-16b8d1b023b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:12.073608 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 67/143] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:12.364989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f192f-db9e-4335-9f17-5f69bf879564 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:12.523974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:12.534629 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f192f-db9e-4335-9f17-5f69bf879564 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:12.581586 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a50619-0520-44aa-9a7e-766020ec43a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:12.582639 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a50619-0520-44aa-9a7e-766020ec43a9 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:12.583195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a50619-0520-44aa-9a7e-766020ec43a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:12.583482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a50619-0520-44aa-9a7e-766020ec43a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:12.602349 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a50619-0520-44aa-9a7e-766020ec43a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:12.607602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-30a50619-0520-44aa-9a7e-766020ec43a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:12.607602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:12.607602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:12.607602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:12.607602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:12.607602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:12.607602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:12.607602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:12.607602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:12.607602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:12.607602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:12.607602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:12.607602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:12.607602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:12.607602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:12.607602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:12.608526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:12.608526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:12.608526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:12.608526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:12.608526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:12.608526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:12.608526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:12.608526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:12.608526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:12.608526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:12.608526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:12.608526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:12.608526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:12.608526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:12.608526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:12.608526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:12.609321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:12.609321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:12.609321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:12.609321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:12.609321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:12.609321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:12.609321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:12.609321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:12.609321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:12.609321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:12.609321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:12.609321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:12.609321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 77/144] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:12.620169 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b302d9c5-34ff-4440-a75f-16b8d1b023b3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:12.772193 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b302d9c5-34ff-4440-a75f-16b8d1b023b3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:12.775646 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:12.818924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74833dd8-36ac-4365-9fdb-35623e58cf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:12.819920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74833dd8-36ac-4365-9fdb-35623e58cf69 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:12.820513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74833dd8-36ac-4365-9fdb-35623e58cf69 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:12.821004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74833dd8-36ac-4365-9fdb-35623e58cf69 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:12.822622 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74833dd8-36ac-4365-9fdb-35623e58cf69 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:12.823265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74833dd8-36ac-4365-9fdb-35623e58cf69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:12.824927 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74833dd8-36ac-4365-9fdb-35623e58cf69 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:12.835109 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 68/145] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:12.843188 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a50619-0520-44aa-9a7e-766020ec43a9 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:13.020204 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a50619-0520-44aa-9a7e-766020ec43a9 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:13.044814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb66b8d-4406-47b2-a4b7-e99f006cfa3e None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:13.045962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb66b8d-4406-47b2-a4b7-e99f006cfa3e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:13.046675 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb66b8d-4406-47b2-a4b7-e99f006cfa3e None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:13.048417 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 69/146] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:53:13.057865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa0282-ca9c-437e-9277-69963ba4b9a9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:13.058524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa0282-ca9c-437e-9277-69963ba4b9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:13.059062 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa0282-ca9c-437e-9277-69963ba4b9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:13.081779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-38aa0282-ca9c-437e-9277-69963ba4b9a9 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:13.083472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-38aa0282-ca9c-437e-9277-69963ba4b9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:13.093199 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824d00b-cbf1-47e9-845a-0e7b62184022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:13.094516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824d00b-cbf1-47e9-845a-0e7b62184022 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:13.094811 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824d00b-cbf1-47e9-845a-0e7b62184022 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:13.095146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824d00b-cbf1-47e9-845a-0e7b62184022 None admin] PATH_INFO: `/v3/users/ffb9e4ff6b76404b9d9afd02933c035e` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:13.097110 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824d00b-cbf1-47e9-845a-0e7b62184022 None admin] RBAC: Authorizing `identity:get_user(user_id=ffb9e4ff6b76404b9d9afd02933c035e)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:13.099705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824d00b-cbf1-47e9-845a-0e7b62184022 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:13.108563 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 78/147] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:12 2022] GET /identity/v3/users/ffb9e4ff6b76404b9d9afd02933c035e => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:13.113149 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824d00b-cbf1-47e9-845a-0e7b62184022 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:13.172314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:13.208422 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 70/148] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:13.285735 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ffe5b-b497-49d3-815f-ba7e95e7a68b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:13.286342 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ffe5b-b497-49d3-815f-ba7e95e7a68b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:13.286834 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ffe5b-b497-49d3-815f-ba7e95e7a68b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:13.288452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 71/149] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:53:13.297528 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bac390-ee80-4f4f-8fb7-72b94733ab49 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:13.298307 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bac390-ee80-4f4f-8fb7-72b94733ab49 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:13.298775 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bac390-ee80-4f4f-8fb7-72b94733ab49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:13.330464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-56bac390-ee80-4f4f-8fb7-72b94733ab49 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:13.332181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-56bac390-ee80-4f4f-8fb7-72b94733ab49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:13.339909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824d00b-cbf1-47e9-845a-0e7b62184022 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:13.425279 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368d687-eb56-452b-85bb-4ea6febee008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:13.426321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368d687-eb56-452b-85bb-4ea6febee008 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:13.426612 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368d687-eb56-452b-85bb-4ea6febee008 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:13.426889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368d687-eb56-452b-85bb-4ea6febee008 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:13.492063 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 72/150] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:13.549070 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368d687-eb56-452b-85bb-4ea6febee008 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:13.550608 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368d687-eb56-452b-85bb-4ea6febee008 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:13.583050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:13.626816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bac390-ee80-4f4f-8fb7-72b94733ab49 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:13.666972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:13.707640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 79/151] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:13.714556 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368d687-eb56-452b-85bb-4ea6febee008 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:13.763818 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bac390-ee80-4f4f-8fb7-72b94733ab49 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:13.828395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b20bda-446d-42da-9b8b-3cbb5d249fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:13.828976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b20bda-446d-42da-9b8b-3cbb5d249fbb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:13.829256 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b20bda-446d-42da-9b8b-3cbb5d249fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:13.829529 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b20bda-446d-42da-9b8b-3cbb5d249fbb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:13.855923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b20bda-446d-42da-9b8b-3cbb5d249fbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:13.860885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e5b20bda-446d-42da-9b8b-3cbb5d249fbb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:13.860885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:13.860885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:13.860885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:13.860885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:13.860885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:13.860885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:13.860885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:13.860885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:13.860885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:13.860885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:13.860885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:13.860885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:13.860885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:13.860885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:13.860885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:13.861759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:13.861759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:13.861759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:13.861759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:13.861759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:13.861759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:13.861759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:13.861759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:13.861759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:13.861759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:13.861759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:13.861759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:13.861759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:13.861759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:13.861759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:13.861759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:13.862540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:13.862540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:13.862540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:13.862540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:13.862540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:13.862540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:13.862540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:13.862540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:13.862540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:13.862540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:13.862540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:13.862540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:13.862540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 73/152] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:53:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:53:13.870577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b20bda-446d-42da-9b8b-3cbb5d249fbb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:13.914442 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368d687-eb56-452b-85bb-4ea6febee008 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:13.952051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:13.970244 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867edb3-e958-4012-8312-bfeebcb1bf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:13.971279 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867edb3-e958-4012-8312-bfeebcb1bf45 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:13.971802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867edb3-e958-4012-8312-bfeebcb1bf45 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:13.974693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867edb3-e958-4012-8312-bfeebcb1bf45 None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:13.976130 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867edb3-e958-4012-8312-bfeebcb1bf45 None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:13.977816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867edb3-e958-4012-8312-bfeebcb1bf45 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:13.980760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 80/153] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:13 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:13.986688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867edb3-e958-4012-8312-bfeebcb1bf45 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:14.015441 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:14.035176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b20bda-446d-42da-9b8b-3cbb5d249fbb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:14.100416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74274972-e8e3-41a6-9fb8-bd725290705f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:14.100966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74274972-e8e3-41a6-9fb8-bd725290705f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:14.101246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74274972-e8e3-41a6-9fb8-bd725290705f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:14.101518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74274972-e8e3-41a6-9fb8-bd725290705f None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:14.102201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74274972-e8e3-41a6-9fb8-bd725290705f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:14.102551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74274972-e8e3-41a6-9fb8-bd725290705f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:14.104485 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74274972-e8e3-41a6-9fb8-bd725290705f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:14.113288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 74/154] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:53:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:14.120239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74274972-e8e3-41a6-9fb8-bd725290705f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:14.193670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867edb3-e958-4012-8312-bfeebcb1bf45 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:14.280506 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48089088-a2a9-4640-b202-e4ef1dc47755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:14.281550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48089088-a2a9-4640-b202-e4ef1dc47755 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:14.282294 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48089088-a2a9-4640-b202-e4ef1dc47755 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:14.282855 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48089088-a2a9-4640-b202-e4ef1dc47755 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:14.285329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48089088-a2a9-4640-b202-e4ef1dc47755 None admin] RBAC: Adding query filter params (role.id=dd5cafa908a74cfd8e1a8baa44122242, scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45, user.id=ffb9e4ff6b76404b9d9afd02933c035e) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:14.285844 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48089088-a2a9-4640-b202-e4ef1dc47755 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:14.287815 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48089088-a2a9-4640-b202-e4ef1dc47755 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:14.301103 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 81/155] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:13 2022] GET /identity/v3/role_assignments?user.id=ffb9e4ff6b76404b9d9afd02933c035e&scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45&role.id=dd5cafa908a74cfd8e1a8baa44122242 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:14.303437 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48089088-a2a9-4640-b202-e4ef1dc47755 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:14.334519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74274972-e8e3-41a6-9fb8-bd725290705f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:14.413738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a5134-3529-4afd-bd9f-4d7d2e7e81cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:14.414305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a5134-3529-4afd-bd9f-4d7d2e7e81cc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:14.414589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a5134-3529-4afd-bd9f-4d7d2e7e81cc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:14.414862 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a5134-3529-4afd-bd9f-4d7d2e7e81cc None admin] PATH_INFO: `/v3/users/ffb9e4ff6b76404b9d9afd02933c035e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:14.422751 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a5134-3529-4afd-bd9f-4d7d2e7e81cc None admin] RBAC: Authorizing `identity:get_user(user_id=ffb9e4ff6b76404b9d9afd02933c035e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:14.422751 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a5134-3529-4afd-bd9f-4d7d2e7e81cc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:14.423128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 75/156] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:14 2022] GET /identity/v3/users/ffb9e4ff6b76404b9d9afd02933c035e => generated 275 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:14.424383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:14.430716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a5134-3529-4afd-bd9f-4d7d2e7e81cc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:14.472101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:14.500581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48089088-a2a9-4640-b202-e4ef1dc47755 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:14.563511 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e251c-730b-46d4-a125-276cc1fcd2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:14.564540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e251c-730b-46d4-a125-276cc1fcd2eb None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:14.565027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e251c-730b-46d4-a125-276cc1fcd2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:14.565505 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e251c-730b-46d4-a125-276cc1fcd2eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:14.593691 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e251c-730b-46d4-a125-276cc1fcd2eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:14.605197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-189e251c-730b-46d4-a125-276cc1fcd2eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:14.605197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:14.605197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:14.605197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:14.605197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:14.605197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:14.605197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:14.605197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:14.605197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:14.605197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:14.605197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:14.605197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:14.605197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:14.605197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:14.605197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:14.605197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:14.606151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:14.606151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:14.606151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:14.606151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:14.606151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:14.606151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:14.606151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:14.606151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:14.606151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:14.606151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:14.606151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:14.606151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:14.606151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:14.606151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:14.606151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:14.606151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:14.606984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:14.606984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:14.606984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:14.606984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:14.606984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:14.606984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:14.606984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:14.606984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:14.606984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:14.606984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:14.606984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:14.606984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:14.609717 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 82/157] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:14.618767 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e251c-730b-46d4-a125-276cc1fcd2eb None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:14.621941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a5134-3529-4afd-bd9f-4d7d2e7e81cc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:14.687034 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85748c-a1d6-4a15-8aaa-a228d268d3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:14.687653 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85748c-a1d6-4a15-8aaa-a228d268d3ac None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:14.687875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85748c-a1d6-4a15-8aaa-a228d268d3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:14.688196 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85748c-a1d6-4a15-8aaa-a228d268d3ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:14.783942 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85748c-a1d6-4a15-8aaa-a228d268d3ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:14.785457 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85748c-a1d6-4a15-8aaa-a228d268d3ac None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:14.795157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e251c-730b-46d4-a125-276cc1fcd2eb None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:14.802370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:14.817373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:14.850827 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0992602-4d3b-4fc9-a4ec-0dbaf9bfb02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:14.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0992602-4d3b-4fc9-a4ec-0dbaf9bfb02e None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:14.852531 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0992602-4d3b-4fc9-a4ec-0dbaf9bfb02e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:14.853183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0992602-4d3b-4fc9-a4ec-0dbaf9bfb02e None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:14.854135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0992602-4d3b-4fc9-a4ec-0dbaf9bfb02e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:14.854761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0992602-4d3b-4fc9-a4ec-0dbaf9bfb02e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:14.856418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0992602-4d3b-4fc9-a4ec-0dbaf9bfb02e None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:14.871760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 83/158] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:14.878172 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0992602-4d3b-4fc9-a4ec-0dbaf9bfb02e None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:14.955044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 76/159] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 07:53:14.956275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc02c8-8aad-410a-a9c6-2d6c0e36bbe2 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:14.956542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc02c8-8aad-410a-a9c6-2d6c0e36bbe2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:14.956814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc02c8-8aad-410a-a9c6-2d6c0e36bbe2 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:14.958098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 77/160] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:14.968372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646e805-750b-4833-9eb7-0f4dd88d2b35 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:14.968652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646e805-750b-4833-9eb7-0f4dd88d2b35 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:14.968920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646e805-750b-4833-9eb7-0f4dd88d2b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:15.011996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9646e805-750b-4833-9eb7-0f4dd88d2b35 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:15.013225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9646e805-750b-4833-9eb7-0f4dd88d2b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:15.024586 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0992602-4d3b-4fc9-a4ec-0dbaf9bfb02e None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:15.065547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a62479-9e0e-4bd3-a07c-d076de81c511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:15.066643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a62479-9e0e-4bd3-a07c-d076de81c511 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:15.067158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a62479-9e0e-4bd3-a07c-d076de81c511 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:15.067753 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a62479-9e0e-4bd3-a07c-d076de81c511 None admin] PATH_INFO: `/v3/users/b351dfc51ec94ee29ccd78418e90d247` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:15.069301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a62479-9e0e-4bd3-a07c-d076de81c511 None admin] RBAC: Authorizing `identity:get_user(user_id=b351dfc51ec94ee29ccd78418e90d247)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:15.071104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a62479-9e0e-4bd3-a07c-d076de81c511 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:15.073353 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 84/161] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:14 2022] GET /identity/v3/users/b351dfc51ec94ee29ccd78418e90d247 => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:15.074913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cac3c-19a7-444f-8cba-1b4b4140bf00 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:15.079768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cac3c-19a7-444f-8cba-1b4b4140bf00 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:15.080299 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cac3c-19a7-444f-8cba-1b4b4140bf00 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:15.081963 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 85/162] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:15.082892 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cac3c-19a7-444f-8cba-1b4b4140bf00 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:15.126391 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 78/163] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:15.127450 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d153b-0eb9-4f1a-8a7c-8ecd8f2468f6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:15.127732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d153b-0eb9-4f1a-8a7c-8ecd8f2468f6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:15.128058 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d153b-0eb9-4f1a-8a7c-8ecd8f2468f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:15.145311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:15.167988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-929d153b-0eb9-4f1a-8a7c-8ecd8f2468f6 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:15.169195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-929d153b-0eb9-4f1a-8a7c-8ecd8f2468f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:15.217906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cac3c-19a7-444f-8cba-1b4b4140bf00 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:15.239313 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:15.260186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ccb70-3ea3-456b-b879-1d44e7ed81dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:15.260821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ccb70-3ea3-456b-b879-1d44e7ed81dc None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:15.261100 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ccb70-3ea3-456b-b879-1d44e7ed81dc None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:15.261410 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ccb70-3ea3-456b-b879-1d44e7ed81dc None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:15.262644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ccb70-3ea3-456b-b879-1d44e7ed81dc None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:15.264267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ccb70-3ea3-456b-b879-1d44e7ed81dc None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:15.266045 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 86/164] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:15 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:15.266767 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ccb70-3ea3-456b-b879-1d44e7ed81dc None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:15.280766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 79/165] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:15.290000 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d153b-0eb9-4f1a-8a7c-8ecd8f2468f6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:15.384493 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ccb70-3ea3-456b-b879-1d44e7ed81dc None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:15.404591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:15.440734 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb26a1-6366-40da-b7c7-da4804cd2246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:15.441365 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb26a1-6366-40da-b7c7-da4804cd2246 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:15.441635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb26a1-6366-40da-b7c7-da4804cd2246 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:15.441897 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb26a1-6366-40da-b7c7-da4804cd2246 None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:15.443052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb26a1-6366-40da-b7c7-da4804cd2246 None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:15.444499 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb26a1-6366-40da-b7c7-da4804cd2246 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:15.446416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 87/166] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:15 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:15.453212 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb26a1-6366-40da-b7c7-da4804cd2246 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:15.456689 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d153b-0eb9-4f1a-8a7c-8ecd8f2468f6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:15.522580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862aeb4-78d7-44d9-bd21-b34d19a97501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:15.523224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862aeb4-78d7-44d9-bd21-b34d19a97501 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:15.523506 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862aeb4-78d7-44d9-bd21-b34d19a97501 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:15.523776 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862aeb4-78d7-44d9-bd21-b34d19a97501 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:15.524513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862aeb4-78d7-44d9-bd21-b34d19a97501 None admin] RBAC: Adding query filter params (role.id=d9b2237fd9884cc39771e75bf902e203, scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45, user.id=ffb9e4ff6b76404b9d9afd02933c035e) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:15.525413 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862aeb4-78d7-44d9-bd21-b34d19a97501 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:15.526835 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862aeb4-78d7-44d9-bd21-b34d19a97501 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:15.541838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 80/167] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:15 2022] GET /identity/v3/role_assignments?user.id=ffb9e4ff6b76404b9d9afd02933c035e&scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45&role.id=d9b2237fd9884cc39771e75bf902e203 => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:15.614387 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb26a1-6366-40da-b7c7-da4804cd2246 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:15.632203 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862aeb4-78d7-44d9-bd21-b34d19a97501 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:15.657634 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ba88b-2add-4a92-b59e-f498430405f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:15.658360 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ba88b-2add-4a92-b59e-f498430405f6 None admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:15.658675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ba88b-2add-4a92-b59e-f498430405f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:15.658963 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ba88b-2add-4a92-b59e-f498430405f6 None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45/users/b351dfc51ec94ee29ccd78418e90d247/roles/74955b19ba90422cbe29fa461c3f287f` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:15.660921 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ba88b-2add-4a92-b59e-f498430405f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45, user_id=b351dfc51ec94ee29ccd78418e90d247, role_id=74955b19ba90422cbe29fa461c3f287f)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:15.662368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ba88b-2add-4a92-b59e-f498430405f6 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:15.716311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:15.767657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 88/168] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:53:15 2022] PUT /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45/users/b351dfc51ec94ee29ccd78418e90d247/roles/74955b19ba90422cbe29fa461c3f287f => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:53:15.785753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862aeb4-78d7-44d9-bd21-b34d19a97501 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:15.841843 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20646ae-d54f-47a6-8665-ae42308da8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:15.842505 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20646ae-d54f-47a6-8665-ae42308da8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:15.842783 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20646ae-d54f-47a6-8665-ae42308da8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:15.843081 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20646ae-d54f-47a6-8665-ae42308da8b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:15.856689 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20646ae-d54f-47a6-8665-ae42308da8b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:15.861920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-b20646ae-d54f-47a6-8665-ae42308da8b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:15.861920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:15.861920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:15.861920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:15.861920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:15.861920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:15.861920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:15.861920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:15.861920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:15.861920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:15.861920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:15.861920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:15.861920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:15.861920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:15.861920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:15.861920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:15.863677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:15.863677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:15.863677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:15.863677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:15.863677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:15.863677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:15.863677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:15.863677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:15.863677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:15.863677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:15.863677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:15.863677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:15.867096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 81/169] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:53:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:15.872176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ba88b-2add-4a92-b59e-f498430405f6 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:15.881085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20646ae-d54f-47a6-8665-ae42308da8b3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:15.950616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:16.047406 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:16.052136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20646ae-d54f-47a6-8665-ae42308da8b3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:16.095689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ba88b-2add-4a92-b59e-f498430405f6 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:16.098802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450e066-0e30-44be-a263-1d53f69040a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:16.099518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450e066-0e30-44be-a263-1d53f69040a8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:16.099778 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450e066-0e30-44be-a263-1d53f69040a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:16.100045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450e066-0e30-44be-a263-1d53f69040a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:16.119203 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450e066-0e30-44be-a263-1d53f69040a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:16.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1450e066-0e30-44be-a263-1d53f69040a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:16.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:16.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:16.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:16.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:16.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:16.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:16.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:16.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:16.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:16.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:16.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:16.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:16.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:16.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:16.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:16.129192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:16.129192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:16.129192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:16.129192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:16.129192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:16.129192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:16.129192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:16.129192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:16.129192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:16.129192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:16.129192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:16.129192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:16.129192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:16.129192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:16.129192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:16.129192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:16.130426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:16.130426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:16.130426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:16.130426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:16.130426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:16.130426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:16.130426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:16.130426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:16.130426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:16.130426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:16.130426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:16.130426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:16.130426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 82/170] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:53:16.138280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450e066-0e30-44be-a263-1d53f69040a8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:16.163293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf5857-28c0-4727-93a4-a80ea88b71a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:16.163936 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf5857-28c0-4727-93a4-a80ea88b71a2 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:16.164187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf5857-28c0-4727-93a4-a80ea88b71a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:16.164452 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf5857-28c0-4727-93a4-a80ea88b71a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:16.165123 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf5857-28c0-4727-93a4-a80ea88b71a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:16.165519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf5857-28c0-4727-93a4-a80ea88b71a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:16.171338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf5857-28c0-4727-93a4-a80ea88b71a2 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:16.183738 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 89/171] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:16.204778 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf5857-28c0-4727-93a4-a80ea88b71a2 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:16.336806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf5857-28c0-4727-93a4-a80ea88b71a2 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:16.338175 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450e066-0e30-44be-a263-1d53f69040a8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:16.342963 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:16.374235 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c98309-af18-44ff-a9b3-6d75896fc3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:16.374867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c98309-af18-44ff-a9b3-6d75896fc3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:16.375162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c98309-af18-44ff-a9b3-6d75896fc3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:16.375432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c98309-af18-44ff-a9b3-6d75896fc3a5 None admin] PATH_INFO: `/v3/users/93747f497a1f4adcb4b8ddb03adfe01c` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:16.376563 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c98309-af18-44ff-a9b3-6d75896fc3a5 None admin] RBAC: Authorizing `identity:get_user(user_id=93747f497a1f4adcb4b8ddb03adfe01c)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:16.378027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c98309-af18-44ff-a9b3-6d75896fc3a5 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:16.379836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 90/172] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:16 2022] GET /identity/v3/users/93747f497a1f4adcb4b8ddb03adfe01c => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:16.385992 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c98309-af18-44ff-a9b3-6d75896fc3a5 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:16.417014 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3d6cc-88ae-40f1-bf2c-28f8802cd747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:16.418763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3d6cc-88ae-40f1-bf2c-28f8802cd747 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:16.419082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3d6cc-88ae-40f1-bf2c-28f8802cd747 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:16.419360 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3d6cc-88ae-40f1-bf2c-28f8802cd747 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:16.420054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3d6cc-88ae-40f1-bf2c-28f8802cd747 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:16.420362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3d6cc-88ae-40f1-bf2c-28f8802cd747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:16.421759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3d6cc-88ae-40f1-bf2c-28f8802cd747 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:16.442276 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 83/173] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:16.443356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c2572-aab6-4b2f-90ac-6efe5cd299f5 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:16.443777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c2572-aab6-4b2f-90ac-6efe5cd299f5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:16.443963 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c2572-aab6-4b2f-90ac-6efe5cd299f5 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:16.445123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 84/174] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:16.450496 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c2572-aab6-4b2f-90ac-6efe5cd299f5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:16.502987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:16.507146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c98309-af18-44ff-a9b3-6d75896fc3a5 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:16.544972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4a7dd-8756-498b-8320-06abe6ffc3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:16.545586 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4a7dd-8756-498b-8320-06abe6ffc3ae None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:16.545857 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4a7dd-8756-498b-8320-06abe6ffc3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:16.546122 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4a7dd-8756-498b-8320-06abe6ffc3ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:16.591350 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:16.619920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4a7dd-8756-498b-8320-06abe6ffc3ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:16.621409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4a7dd-8756-498b-8320-06abe6ffc3ae None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:16.720197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c2572-aab6-4b2f-90ac-6efe5cd299f5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:16.732458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 91/175] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 07:53:16.732458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a959b63-859e-449c-900b-45063df2a525 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:16.732458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a959b63-859e-449c-900b-45063df2a525 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:16.732458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a959b63-859e-449c-900b-45063df2a525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:16.752573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-1a959b63-859e-449c-900b-45063df2a525 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:16.753761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1a959b63-859e-449c-900b-45063df2a525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:16.785162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf80ba5-0a53-4799-82d1-d63962cfcf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:16.786179 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf80ba5-0a53-4799-82d1-d63962cfcf2c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:16.786690 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf80ba5-0a53-4799-82d1-d63962cfcf2c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:16.787206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf80ba5-0a53-4799-82d1-d63962cfcf2c None admin] PATH_INFO: `/v3/users/ffb9e4ff6b76404b9d9afd02933c035e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:16.791792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf80ba5-0a53-4799-82d1-d63962cfcf2c None admin] RBAC: Authorizing `identity:get_user(user_id=ffb9e4ff6b76404b9d9afd02933c035e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:16.793610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf80ba5-0a53-4799-82d1-d63962cfcf2c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:16.795573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 85/176] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:16 2022] GET /identity/v3/users/ffb9e4ff6b76404b9d9afd02933c035e => generated 275 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:16.799672 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf80ba5-0a53-4799-82d1-d63962cfcf2c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:16.839382 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 92/177] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:16.839382 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a959b63-859e-449c-900b-45063df2a525 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:16.839382 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:16.943505 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a959b63-859e-449c-900b-45063df2a525 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:16.955309 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:16.980940 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7deab-1f18-4ba3-9235-33b8220a3ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:16.981550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7deab-1f18-4ba3-9235-33b8220a3ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:16.981859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7deab-1f18-4ba3-9235-33b8220a3ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:16.982131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7deab-1f18-4ba3-9235-33b8220a3ff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:16.988887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf80ba5-0a53-4799-82d1-d63962cfcf2c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:17.054784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7deab-1f18-4ba3-9235-33b8220a3ff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:17.056509 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7deab-1f18-4ba3-9235-33b8220a3ff6 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:17.058416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:17.085051 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf9e91-ed3e-4d63-9c24-50271b5a0f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:17.085698 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf9e91-ed3e-4d63-9c24-50271b5a0f44 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:17.085979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf9e91-ed3e-4d63-9c24-50271b5a0f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:17.086249 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf9e91-ed3e-4d63-9c24-50271b5a0f44 None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:17.091672 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf9e91-ed3e-4d63-9c24-50271b5a0f44 None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:17.097371 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf9e91-ed3e-4d63-9c24-50271b5a0f44 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:17.097371 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 86/178] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:16 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:17.097371 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31496af3-84a0-455f-87e1-56a8c3afed3a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:17.097371 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31496af3-84a0-455f-87e1-56a8c3afed3a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:17.097371 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31496af3-84a0-455f-87e1-56a8c3afed3a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:17.097712 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 87/179] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:17.104571 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a4302-689b-42af-8b45-c9fbc7f6cff8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:17.104851 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a4302-689b-42af-8b45-c9fbc7f6cff8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:17.105129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a4302-689b-42af-8b45-c9fbc7f6cff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:17.148080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-994a4302-689b-42af-8b45-c9fbc7f6cff8 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:17.149295 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-994a4302-689b-42af-8b45-c9fbc7f6cff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:17.176071 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 93/180] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:17.176927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7deab-1f18-4ba3-9235-33b8220a3ff6 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:17.272203 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 88/181] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:17.273550 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a4302-689b-42af-8b45-c9fbc7f6cff8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:17.297767 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7deab-1f18-4ba3-9235-33b8220a3ff6 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:17.299880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:17.339125 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a367f4-ceaa-450f-9755-25a5bf3d2047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:17.339721 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a367f4-ceaa-450f-9755-25a5bf3d2047 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:17.339992 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a367f4-ceaa-450f-9755-25a5bf3d2047 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:17.340257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a367f4-ceaa-450f-9755-25a5bf3d2047 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:17.340980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a367f4-ceaa-450f-9755-25a5bf3d2047 None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45, user.id=93747f497a1f4adcb4b8ddb03adfe01c) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:17.341280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a367f4-ceaa-450f-9755-25a5bf3d2047 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:17.342727 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a367f4-ceaa-450f-9755-25a5bf3d2047 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:17.349966 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 94/182] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:17 2022] GET /identity/v3/role_assignments?user.id=93747f497a1f4adcb4b8ddb03adfe01c&scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:17.350533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a367f4-ceaa-450f-9755-25a5bf3d2047 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:17.416800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:17.450770 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a4302-689b-42af-8b45-c9fbc7f6cff8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:17.511333 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc39c8-99a4-4a9a-beec-4b68c5db9dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:17.511990 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc39c8-99a4-4a9a-beec-4b68c5db9dff None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:17.512268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc39c8-99a4-4a9a-beec-4b68c5db9dff None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:17.512539 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc39c8-99a4-4a9a-beec-4b68c5db9dff None admin] PATH_INFO: `/v3/projects/bd37fdac1af54609810832817bd912d4` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:17.513993 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc39c8-99a4-4a9a-beec-4b68c5db9dff None admin] RBAC: Authorizing `identity:get_project(project_id=bd37fdac1af54609810832817bd912d4)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:17.516549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc39c8-99a4-4a9a-beec-4b68c5db9dff None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:17.517579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 89/183] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:17 2022] GET /identity/v3/projects/bd37fdac1af54609810832817bd912d4 => generated 312 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:17.518568 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90f80e-06bb-41a9-9f0e-1a3887a79949 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:17.518855 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90f80e-06bb-41a9-9f0e-1a3887a79949 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:17.519157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90f80e-06bb-41a9-9f0e-1a3887a79949 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:17.520629 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 90/184] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:17.523613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90f80e-06bb-41a9-9f0e-1a3887a79949 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:17.555736 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a367f4-ceaa-450f-9755-25a5bf3d2047 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:17.603574 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67427c3-8e99-4d74-be28-d610e0a37bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:17.604544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67427c3-8e99-4d74-be28-d610e0a37bef None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:17.605117 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67427c3-8e99-4d74-be28-d610e0a37bef None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:17.605399 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67427c3-8e99-4d74-be28-d610e0a37bef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:17.619819 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67427c3-8e99-4d74-be28-d610e0a37bef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:17.625446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-b67427c3-8e99-4d74-be28-d610e0a37bef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:17.625446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:17.625446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:17.625446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:17.625446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:17.625446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:17.625446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:17.625446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:17.625446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:17.625446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:17.625446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:17.625446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:17.625446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:17.625446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:17.625446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:17.625446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:17.626725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:17.626725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:17.626725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:17.626725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:17.626725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:17.626725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:17.626725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:17.626725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:17.626725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:17.626725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:17.626725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:17.626725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:17.626725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:17.626725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:17.626725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:17.626725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:17.627541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:17.627541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:17.627541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:17.627541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:17.627541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:17.627541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:17.627541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:17.627541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:17.627541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:17.627541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:17.627541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:17.627541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:17.627541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 95/185] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:53:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:17.628182 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228f66b-efc4-488b-9117-397d34b2e3bb None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:17.628254 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228f66b-efc4-488b-9117-397d34b2e3bb None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:17.629104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228f66b-efc4-488b-9117-397d34b2e3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:17.649379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-d228f66b-efc4-488b-9117-397d34b2e3bb None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:17.651168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d228f66b-efc4-488b-9117-397d34b2e3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:17.673354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:17.689655 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90f80e-06bb-41a9-9f0e-1a3887a79949 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:17.723631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 96/186] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:17.724455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228f66b-efc4-488b-9117-397d34b2e3bb None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:17.728735 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb92dc-549d-4cae-8bb5-2fb349abfb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:17.729999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb92dc-549d-4cae-8bb5-2fb349abfb71 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:17.730513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb92dc-549d-4cae-8bb5-2fb349abfb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:17.731015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb92dc-549d-4cae-8bb5-2fb349abfb71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:17.731955 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb92dc-549d-4cae-8bb5-2fb349abfb71 None admin] RBAC: Adding query filter params (role.id=dd5cafa908a74cfd8e1a8baa44122242, scope.project.id=bd37fdac1af54609810832817bd912d4, user.id=ffb9e4ff6b76404b9d9afd02933c035e) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:17.732479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb92dc-549d-4cae-8bb5-2fb349abfb71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:17.734668 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb92dc-549d-4cae-8bb5-2fb349abfb71 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:17.742570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 91/187] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:17 2022] GET /identity/v3/role_assignments?user.id=ffb9e4ff6b76404b9d9afd02933c035e&scope.project.id=bd37fdac1af54609810832817bd912d4&role.id=dd5cafa908a74cfd8e1a8baa44122242 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:17.744361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb92dc-549d-4cae-8bb5-2fb349abfb71 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:17.768189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:17.935374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb92dc-549d-4cae-8bb5-2fb349abfb71 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:17.951069 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228f66b-efc4-488b-9117-397d34b2e3bb None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:17.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cc12a-9798-4d55-9633-ee90e41269e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:17.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cc12a-9798-4d55-9633-ee90e41269e0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:17.985695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cc12a-9798-4d55-9633-ee90e41269e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:17.985937 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cc12a-9798-4d55-9633-ee90e41269e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:17.986612 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cc12a-9798-4d55-9633-ee90e41269e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:17.986913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cc12a-9798-4d55-9633-ee90e41269e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:17.988441 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cc12a-9798-4d55-9633-ee90e41269e0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:17.998321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 92/188] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:18.005143 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cc12a-9798-4d55-9633-ee90e41269e0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:18.016434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e46f73-fa02-44eb-8db6-4c1a09543207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:18.018899 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e46f73-fa02-44eb-8db6-4c1a09543207 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:18.018899 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e46f73-fa02-44eb-8db6-4c1a09543207 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:18.018899 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e46f73-fa02-44eb-8db6-4c1a09543207 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:18.033996 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e46f73-fa02-44eb-8db6-4c1a09543207 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:18.041195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-97e46f73-fa02-44eb-8db6-4c1a09543207 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:18.041195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:18.041195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:18.041195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:18.041195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:18.041195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:18.041195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:18.041195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:18.041195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:18.041195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:18.041195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:18.041195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:18.041195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:18.041195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:18.041195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:18.041195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:18.042042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:18.042042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:18.042042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:18.042042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:18.042042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:18.042042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:18.042042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:18.042042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:18.042042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:18.042042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:18.042042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:18.042042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:18.042042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:18.042042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:18.042042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:18.042042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:18.042835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:18.042835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:18.042835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:18.042835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:18.042835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:18.042835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:18.042835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:18.042835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:18.042835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:18.042835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:18.042835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:18.042835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:18.042835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 97/189] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:53:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:18.042835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e46f73-fa02-44eb-8db6-4c1a09543207 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:18.072838 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:18.102481 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:18.122368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cc12a-9798-4d55-9633-ee90e41269e0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:18.161519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e46f73-fa02-44eb-8db6-4c1a09543207 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:18.162866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1b0f8-0f59-4ae7-9157-2aa90a892b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:18.163588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1b0f8-0f59-4ae7-9157-2aa90a892b06 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:18.163955 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1b0f8-0f59-4ae7-9157-2aa90a892b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:18.164292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1b0f8-0f59-4ae7-9157-2aa90a892b06 None admin] PATH_INFO: `/v3/users/24b1ed517c454625a2a733a06aa4fb70` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:18.178554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1b0f8-0f59-4ae7-9157-2aa90a892b06 None admin] RBAC: Authorizing `identity:get_user(user_id=24b1ed517c454625a2a733a06aa4fb70)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:18.180060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1b0f8-0f59-4ae7-9157-2aa90a892b06 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:18.181750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 93/190] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:18 2022] GET /identity/v3/users/24b1ed517c454625a2a733a06aa4fb70 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:18.182510 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1b0f8-0f59-4ae7-9157-2aa90a892b06 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:18.201524 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59c8e7-1b88-47c0-aa33-6f7649fd11da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:18.202082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59c8e7-1b88-47c0-aa33-6f7649fd11da None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:18.202354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59c8e7-1b88-47c0-aa33-6f7649fd11da None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:18.202623 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59c8e7-1b88-47c0-aa33-6f7649fd11da None admin] PATH_INFO: `/v3/domains/default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:18.203763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59c8e7-1b88-47c0-aa33-6f7649fd11da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:18.205210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59c8e7-1b88-47c0-aa33-6f7649fd11da None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:18.206922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 98/191] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:53:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:18.207612 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59c8e7-1b88-47c0-aa33-6f7649fd11da None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:18.293189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1b0f8-0f59-4ae7-9157-2aa90a892b06 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:18.298062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:18.314916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59c8e7-1b88-47c0-aa33-6f7649fd11da None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:18.332123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109184fe-0f54-4d17-85e2-ed13d1e6f591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:18.332722 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109184fe-0f54-4d17-85e2-ed13d1e6f591 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:18.332999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109184fe-0f54-4d17-85e2-ed13d1e6f591 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:18.333268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109184fe-0f54-4d17-85e2-ed13d1e6f591 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:18.334056 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109184fe-0f54-4d17-85e2-ed13d1e6f591 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:18.334358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109184fe-0f54-4d17-85e2-ed13d1e6f591 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:18.335905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109184fe-0f54-4d17-85e2-ed13d1e6f591 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:18.342196 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:18.347246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 94/192] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:53:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:18.347913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109184fe-0f54-4d17-85e2-ed13d1e6f591 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:18.356058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a926e-35e6-4c5d-90f6-da2b2f27fbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:18.356673 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a926e-35e6-4c5d-90f6-da2b2f27fbf8 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:18.356949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a926e-35e6-4c5d-90f6-da2b2f27fbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:18.357220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a926e-35e6-4c5d-90f6-da2b2f27fbf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:18.438108 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a926e-35e6-4c5d-90f6-da2b2f27fbf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:18.439936 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a926e-35e6-4c5d-90f6-da2b2f27fbf8 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:18.460339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109184fe-0f54-4d17-85e2-ed13d1e6f591 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:18.497582 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ab056-4657-49ce-a72d-300bfbb6e288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:18.498186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ab056-4657-49ce-a72d-300bfbb6e288 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:18.498724 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ab056-4657-49ce-a72d-300bfbb6e288 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:18.499022 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ab056-4657-49ce-a72d-300bfbb6e288 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:18.499682 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ab056-4657-49ce-a72d-300bfbb6e288 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:18.501128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ab056-4657-49ce-a72d-300bfbb6e288 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:18.529177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:18.545888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 99/193] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:18.546704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a926e-35e6-4c5d-90f6-da2b2f27fbf8 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:18.592238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 95/194] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:53:18 2022] POST /identity/v3/users => generated 297 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:53:18.592238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ab056-4657-49ce-a72d-300bfbb6e288 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:18.673122 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a926e-35e6-4c5d-90f6-da2b2f27fbf8 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:18.687955 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:18.715457 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334b9f4-6a77-4c9e-acde-0bff0d7123cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:18.716103 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334b9f4-6a77-4c9e-acde-0bff0d7123cb None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:18.716359 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334b9f4-6a77-4c9e-acde-0bff0d7123cb None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:18.716664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334b9f4-6a77-4c9e-acde-0bff0d7123cb None admin] PATH_INFO: `/v3/users/24b1ed517c454625a2a733a06aa4fb70` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:18.717861 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334b9f4-6a77-4c9e-acde-0bff0d7123cb None admin] RBAC: Authorizing `identity:get_user(user_id=24b1ed517c454625a2a733a06aa4fb70)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:18.719652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334b9f4-6a77-4c9e-acde-0bff0d7123cb None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:18.721109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 100/195] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:18 2022] GET /identity/v3/users/24b1ed517c454625a2a733a06aa4fb70 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:18.727189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5253b-6d68-4aca-a8c3-f187ed95e5e3 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:18.727469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5253b-6d68-4aca-a8c3-f187ed95e5e3 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:18.728288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5253b-6d68-4aca-a8c3-f187ed95e5e3 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:18.729723 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 101/196] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:18.730477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5253b-6d68-4aca-a8c3-f187ed95e5e3 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:18.744849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ab056-4657-49ce-a72d-300bfbb6e288 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:18.786725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab120bf-928e-4b68-8130-0f6db8148718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:18.787387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab120bf-928e-4b68-8130-0f6db8148718 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:18.787668 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab120bf-928e-4b68-8130-0f6db8148718 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:18.787939 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab120bf-928e-4b68-8130-0f6db8148718 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:18.789703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab120bf-928e-4b68-8130-0f6db8148718 None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:18.791208 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:18.791326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab120bf-928e-4b68-8130-0f6db8148718 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:18.797702 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 96/197] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:18 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:18.797702 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b1008-7425-4c37-bd20-d0579755aabe None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:18.797702 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b1008-7425-4c37-bd20-d0579755aabe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:18.797702 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b1008-7425-4c37-bd20-d0579755aabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:18.816185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-886b1008-7425-4c37-bd20-d0579755aabe None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:18.817436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-886b1008-7425-4c37-bd20-d0579755aabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:18.852449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5253b-6d68-4aca-a8c3-f187ed95e5e3 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:18.894989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa131b-39de-4bab-8ecc-cd810e885bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:18.894989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa131b-39de-4bab-8ecc-cd810e885bde None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:18.894989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa131b-39de-4bab-8ecc-cd810e885bde None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:18.895318 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa131b-39de-4bab-8ecc-cd810e885bde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:18.896511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 97/198] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:18.897353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b1008-7425-4c37-bd20-d0579755aabe None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:18.966078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:19.004370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa131b-39de-4bab-8ecc-cd810e885bde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:19.006231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa131b-39de-4bab-8ecc-cd810e885bde None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:19.018280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b1008-7425-4c37-bd20-d0579755aabe None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:19.056593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e0db6-2700-492c-b069-39cad5fcd06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:19.057201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e0db6-2700-492c-b069-39cad5fcd06e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:19.057476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e0db6-2700-492c-b069-39cad5fcd06e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:19.058585 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e0db6-2700-492c-b069-39cad5fcd06e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:19.059553 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e0db6-2700-492c-b069-39cad5fcd06e None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=39c302b22dd04479bf8305109d613551, user.id=24b1ed517c454625a2a733a06aa4fb70) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:19.060079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e0db6-2700-492c-b069-39cad5fcd06e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:19.062513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e0db6-2700-492c-b069-39cad5fcd06e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:19.068595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 98/199] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:18 2022] GET /identity/v3/role_assignments?user.id=24b1ed517c454625a2a733a06aa4fb70&scope.project.id=39c302b22dd04479bf8305109d613551&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:19.074840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:19.130554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 102/200] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:19.137293 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e0db6-2700-492c-b069-39cad5fcd06e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:19.222009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa131b-39de-4bab-8ecc-cd810e885bde None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:19.291392 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e0db6-2700-492c-b069-39cad5fcd06e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:19.314012 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:19.338547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f058c-4b8d-4275-b008-6e1b9bbc902d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:19.340048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f058c-4b8d-4275-b008-6e1b9bbc902d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:19.340997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f058c-4b8d-4275-b008-6e1b9bbc902d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:19.341275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f058c-4b8d-4275-b008-6e1b9bbc902d None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:19.347470 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f058c-4b8d-4275-b008-6e1b9bbc902d None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:19.348932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f058c-4b8d-4275-b008-6e1b9bbc902d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:19.349364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa131b-39de-4bab-8ecc-cd810e885bde None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:19.352534 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 99/201] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:19 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:19.361115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f058c-4b8d-4275-b008-6e1b9bbc902d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:19.423642 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a413306-711f-40c8-a879-c97c7edd21be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:19.424670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a413306-711f-40c8-a879-c97c7edd21be None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:19.425149 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a413306-711f-40c8-a879-c97c7edd21be None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:19.425618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a413306-711f-40c8-a879-c97c7edd21be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:19.447468 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a413306-711f-40c8-a879-c97c7edd21be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:19.453555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-9a413306-711f-40c8-a879-c97c7edd21be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:19.453555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:19.453555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:19.453555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:19.453555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:19.453555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:19.453555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:19.453555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:19.453555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:19.453555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:19.453555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:19.453555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:19.453555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:19.453555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:19.453555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:19.453555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:19.454429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:19.454429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:19.454429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:19.454429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:19.454429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:19.454429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:19.454429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:19.454429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:19.454429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:19.454429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:19.454429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:19.454429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:19.454429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:19.454429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:19.454429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:19.454429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:19.455267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:19.455267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:19.455267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:19.455267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:19.455267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:19.455267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:19.455267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:19.455267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:19.455267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:19.455267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:19.455267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:19.455267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:19.460224 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 103/202] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:19.480558 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a413306-711f-40c8-a879-c97c7edd21be None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:19.515901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:19.518222 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f058c-4b8d-4275-b008-6e1b9bbc902d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:19.569856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a53e47-c4a3-434a-9e91-da63ba539513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:19.570469 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a53e47-c4a3-434a-9e91-da63ba539513 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:19.570746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a53e47-c4a3-434a-9e91-da63ba539513 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:19.571046 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a53e47-c4a3-434a-9e91-da63ba539513 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:19.572386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a53e47-c4a3-434a-9e91-da63ba539513 None admin] RBAC: Adding query filter params (role.id=d9b2237fd9884cc39771e75bf902e203, scope.project.id=39c302b22dd04479bf8305109d613551, user.id=24b1ed517c454625a2a733a06aa4fb70) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:19.572693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a53e47-c4a3-434a-9e91-da63ba539513 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:19.574153 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a53e47-c4a3-434a-9e91-da63ba539513 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:19.595147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 100/203] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:19 2022] GET /identity/v3/role_assignments?user.id=24b1ed517c454625a2a733a06aa4fb70&scope.project.id=39c302b22dd04479bf8305109d613551&role.id=d9b2237fd9884cc39771e75bf902e203 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:19.675127 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a413306-711f-40c8-a879-c97c7edd21be None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:19.733921 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45f826-171f-4fce-bc3b-7af1a4612831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:19.734454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45f826-171f-4fce-bc3b-7af1a4612831 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:19.734731 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45f826-171f-4fce-bc3b-7af1a4612831 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:19.735031 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45f826-171f-4fce-bc3b-7af1a4612831 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:19.735727 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45f826-171f-4fce-bc3b-7af1a4612831 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:19.736027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45f826-171f-4fce-bc3b-7af1a4612831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:19.737468 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45f826-171f-4fce-bc3b-7af1a4612831 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:19.752571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 104/204] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:19.758766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a53e47-c4a3-434a-9e91-da63ba539513 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:19.825653 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:19.879555 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a53e47-c4a3-434a-9e91-da63ba539513 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:19.913668 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7f753-330e-4ff4-9d87-fbdccf65abd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:19.914303 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7f753-330e-4ff4-9d87-fbdccf65abd8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:19.914584 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7f753-330e-4ff4-9d87-fbdccf65abd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:19.914858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7f753-330e-4ff4-9d87-fbdccf65abd8 None admin] PATH_INFO: `/v3/users/b351dfc51ec94ee29ccd78418e90d247` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:19.916440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7f753-330e-4ff4-9d87-fbdccf65abd8 None admin] RBAC: Authorizing `identity:get_user(user_id=b351dfc51ec94ee29ccd78418e90d247)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:19.917897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7f753-330e-4ff4-9d87-fbdccf65abd8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:19.919926 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 101/205] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:19 2022] GET /identity/v3/users/b351dfc51ec94ee29ccd78418e90d247 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:19.925484 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45f826-171f-4fce-bc3b-7af1a4612831 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:20.046797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:20.082540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45f826-171f-4fce-bc3b-7af1a4612831 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:20.132980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e531cd6-ee00-4ba6-a553-6abf1e348348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:20.133590 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e531cd6-ee00-4ba6-a553-6abf1e348348 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:20.134070 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e531cd6-ee00-4ba6-a553-6abf1e348348 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:20.134358 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e531cd6-ee00-4ba6-a553-6abf1e348348 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:20.220694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e531cd6-ee00-4ba6-a553-6abf1e348348 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:20.222174 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e531cd6-ee00-4ba6-a553-6abf1e348348 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:20.300558 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:20.333490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 105/206] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:20.340217 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7f753-330e-4ff4-9d87-fbdccf65abd8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:20.487188 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7f753-330e-4ff4-9d87-fbdccf65abd8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:20.527373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:20.555045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bafd6-c059-4952-ab64-e896421c610d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:20.555612 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bafd6-c059-4952-ab64-e896421c610d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:20.556706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bafd6-c059-4952-ab64-e896421c610d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:20.556984 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bafd6-c059-4952-ab64-e896421c610d None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:20.563649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bafd6-c059-4952-ab64-e896421c610d None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:20.565120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bafd6-c059-4952-ab64-e896421c610d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:20.566778 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 102/207] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:20 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:20.573644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e531cd6-ee00-4ba6-a553-6abf1e348348 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:20.716104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e531cd6-ee00-4ba6-a553-6abf1e348348 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:20.753184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b7550-7263-43af-936b-0b9eff779123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:20.753782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b7550-7263-43af-936b-0b9eff779123 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:20.754056 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b7550-7263-43af-936b-0b9eff779123 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:20.754379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b7550-7263-43af-936b-0b9eff779123 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:20.755130 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b7550-7263-43af-936b-0b9eff779123 None admin] RBAC: Adding query filter params (role.id=74955b19ba90422cbe29fa461c3f287f, scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45, user.id=b351dfc51ec94ee29ccd78418e90d247) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:20.755434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b7550-7263-43af-936b-0b9eff779123 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:20.756877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b7550-7263-43af-936b-0b9eff779123 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:20.763349 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 106/208] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:20 2022] GET /identity/v3/role_assignments?user.id=b351dfc51ec94ee29ccd78418e90d247&scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45&role.id=74955b19ba90422cbe29fa461c3f287f => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:21.544072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d410e37-940c-4cdb-a458-57c48e0c40aa None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:21.544597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d410e37-940c-4cdb-a458-57c48e0c40aa None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:21.544597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d410e37-940c-4cdb-a458-57c48e0c40aa None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:21.546209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 103/209] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:21.552364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ce439-73af-45ed-98c5-3d09c540eede None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:21.553001 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ce439-73af-45ed-98c5-3d09c540eede None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:21.553467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ce439-73af-45ed-98c5-3d09c540eede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:21.574253 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-721ce439-73af-45ed-98c5-3d09c540eede None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:21.575882 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-721ce439-73af-45ed-98c5-3d09c540eede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:21.611533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:21.652826 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 107/210] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:21.657287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddf991-b80e-4f26-a642-945954e14eba None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:21.657570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddf991-b80e-4f26-a642-945954e14eba None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:21.657845 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddf991-b80e-4f26-a642-945954e14eba None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:21.659239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 104/211] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:21.664436 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7f792-a224-4c5f-b857-419e3e4c0e23 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:21.664943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7f792-a224-4c5f-b857-419e3e4c0e23 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:21.665963 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7f792-a224-4c5f-b857-419e3e4c0e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:21.685694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-73b7f792-a224-4c5f-b857-419e3e4c0e23 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:21.687213 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-73b7f792-a224-4c5f-b857-419e3e4c0e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:21.697263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc4cf6-a148-4896-b088-b9596a0cda88 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:21.697545 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc4cf6-a148-4896-b088-b9596a0cda88 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:21.697819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc4cf6-a148-4896-b088-b9596a0cda88 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:21.699205 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 105/212] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:53:21.704020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc7d85-8201-46fe-8743-dc94ebe5517f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:21.704303 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc7d85-8201-46fe-8743-dc94ebe5517f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:21.704577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc7d85-8201-46fe-8743-dc94ebe5517f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:21.723834 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-52bc7d85-8201-46fe-8743-dc94ebe5517f None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:21.725134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-52bc7d85-8201-46fe-8743-dc94ebe5517f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:21.773446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 108/213] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:21.808715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 106/214] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:21.944773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7f792-a224-4c5f-b857-419e3e4c0e23 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:22.045379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc7d85-8201-46fe-8743-dc94ebe5517f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:22.056421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:22.060259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7f792-a224-4c5f-b857-419e3e4c0e23 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:22.094960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f7e46-293a-4dfa-beca-2c7c4f660230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:22.095594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f7e46-293a-4dfa-beca-2c7c4f660230 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:22.095869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f7e46-293a-4dfa-beca-2c7c4f660230 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:22.096139 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f7e46-293a-4dfa-beca-2c7c4f660230 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:22.106672 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f7e46-293a-4dfa-beca-2c7c4f660230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:22.111456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-a44f7e46-293a-4dfa-beca-2c7c4f660230 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:22.111456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:22.111456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:22.111456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:22.111456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:22.111456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:22.111456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:22.111456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:22.111456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:22.111456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:22.111456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:22.111456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:22.111456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:22.111456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:22.111456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:22.111456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:22.112395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:22.112395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:22.112395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:22.112395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:22.112395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:22.112395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:22.112395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:22.112395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:22.112395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:22.112395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:22.112395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:22.112395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:22.112395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:22.112395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:22.112395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:22.112395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:22.113193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:22.113193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:22.113193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:22.113193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:22.113193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:22.113193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:22.113193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:22.113193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:22.113193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:22.113193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:22.113193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:22.113193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:22.113193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 109/215] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:22.113852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f41d3d-2092-4960-987b-2b284a59ff71 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:22.113925 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f41d3d-2092-4960-987b-2b284a59ff71 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:22.114192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f41d3d-2092-4960-987b-2b284a59ff71 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:22.115490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 110/216] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:22.119724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f41d3d-2092-4960-987b-2b284a59ff71 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:22.160719 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc7d85-8201-46fe-8743-dc94ebe5517f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:22.177338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:22.195533 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0116b-152d-4011-8ee1-f2456a9e8fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:22.196209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0116b-152d-4011-8ee1-f2456a9e8fee None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:22.196686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0116b-152d-4011-8ee1-f2456a9e8fee None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:22.196964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0116b-152d-4011-8ee1-f2456a9e8fee None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:22.198080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0116b-152d-4011-8ee1-f2456a9e8fee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:22.199610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0116b-152d-4011-8ee1-f2456a9e8fee None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:22.201423 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 107/217] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:53:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:22.202578 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0116b-152d-4011-8ee1-f2456a9e8fee None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:22.221306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f41d3d-2092-4960-987b-2b284a59ff71 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:22.256094 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eadfe2-f7ed-4eab-bab2-3f2c45b1f539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:22.256719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eadfe2-f7ed-4eab-bab2-3f2c45b1f539 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:22.256994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eadfe2-f7ed-4eab-bab2-3f2c45b1f539 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:22.257262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eadfe2-f7ed-4eab-bab2-3f2c45b1f539 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:22.267462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eadfe2-f7ed-4eab-bab2-3f2c45b1f539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:22.272647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-b3eadfe2-f7ed-4eab-bab2-3f2c45b1f539 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:22.272647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:22.272647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:22.272647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:22.272647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:22.272647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:22.272647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:22.272647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:22.272647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:22.272647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:22.272647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:22.272647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:22.272647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:22.272647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:22.272647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:22.272647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:22.273694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:22.273694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:22.273694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:22.273694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:22.273694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:22.273694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:22.273694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:22.273694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:22.273694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:22.273694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:22.273694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:22.273694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:22.273694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:22.273694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:22.273694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:22.273694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:22.274766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:22.274766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:22.274766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:22.274766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:22.274766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:22.274766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:22.274766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:22.274766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:22.274766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:22.274766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:22.274766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:22.274766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:22.274766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 111/218] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:53:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:22.275585 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fc730-dfc3-49fe-991b-7422b75f012f None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:22.275585 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fc730-dfc3-49fe-991b-7422b75f012f None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:22.275585 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2fc730-dfc3-49fe-991b-7422b75f012f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:22.299345 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-8b2fc730-dfc3-49fe-991b-7422b75f012f None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:22.300538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-8b2fc730-dfc3-49fe-991b-7422b75f012f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:22.314592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:22.316271 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0116b-152d-4011-8ee1-f2456a9e8fee None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:22.349892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe7434-45cf-4bfe-9506-2eef96b1b4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:22.350499 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe7434-45cf-4bfe-9506-2eef96b1b4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:22.350779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe7434-45cf-4bfe-9506-2eef96b1b4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:22.351087 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe7434-45cf-4bfe-9506-2eef96b1b4e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:22.351779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe7434-45cf-4bfe-9506-2eef96b1b4e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:22.352078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe7434-45cf-4bfe-9506-2eef96b1b4e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:22.354188 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe7434-45cf-4bfe-9506-2eef96b1b4e7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:22.367150 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 108/219] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:22.367150 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe7434-45cf-4bfe-9506-2eef96b1b4e7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:22.369842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 112/220] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:22.369842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2fc730-dfc3-49fe-991b-7422b75f012f None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:22.410798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:22.469279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe7434-45cf-4bfe-9506-2eef96b1b4e7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:22.473618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2fc730-dfc3-49fe-991b-7422b75f012f None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:22.509065 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00caa467-2c1a-4979-903c-8f82c4a9b7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:22.509753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00caa467-2c1a-4979-903c-8f82c4a9b7cd None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:22.510025 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15906b1-8eb6-4086-a1bb-afb612a83d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:22.510138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00caa467-2c1a-4979-903c-8f82c4a9b7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:22.510316 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00caa467-2c1a-4979-903c-8f82c4a9b7cd None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:22.510631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15906b1-8eb6-4086-a1bb-afb612a83d86 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:22.510909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15906b1-8eb6-4086-a1bb-afb612a83d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:22.511017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00caa467-2c1a-4979-903c-8f82c4a9b7cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:22.511226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15906b1-8eb6-4086-a1bb-afb612a83d86 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:22.511895 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15906b1-8eb6-4086-a1bb-afb612a83d86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:22.512196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15906b1-8eb6-4086-a1bb-afb612a83d86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:22.512548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00caa467-2c1a-4979-903c-8f82c4a9b7cd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:22.513568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15906b1-8eb6-4086-a1bb-afb612a83d86 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:22.521920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 113/221] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:22.527288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15906b1-8eb6-4086-a1bb-afb612a83d86 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:22.540697 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:22.619105 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 109/222] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:53:22 2022] POST /identity/v3/users => generated 297 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:53:22.619863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00caa467-2c1a-4979-903c-8f82c4a9b7cd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:22.646657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15906b1-8eb6-4086-a1bb-afb612a83d86 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:22.684577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19510c-9bd8-4738-88c1-ed0aa5e1d6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:22.685236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19510c-9bd8-4738-88c1-ed0aa5e1d6fa None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:22.685516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19510c-9bd8-4738-88c1-ed0aa5e1d6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:22.685789 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19510c-9bd8-4738-88c1-ed0aa5e1d6fa None admin] PATH_INFO: `/v3/users/111855efac094907a56d3a29d5f61ee8` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:22.700342 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19510c-9bd8-4738-88c1-ed0aa5e1d6fa None admin] RBAC: Authorizing `identity:get_user(user_id=111855efac094907a56d3a29d5f61ee8)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:22.701848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19510c-9bd8-4738-88c1-ed0aa5e1d6fa None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:22.703626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 114/223] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:22 2022] GET /identity/v3/users/111855efac094907a56d3a29d5f61ee8 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:22.708417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19510c-9bd8-4738-88c1-ed0aa5e1d6fa None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:22.753545 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00caa467-2c1a-4979-903c-8f82c4a9b7cd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:22.755553 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:22.778987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:22.798758 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3addd593-0b4c-4e7f-9b1f-623cc2b1ba76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:22.799683 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3addd593-0b4c-4e7f-9b1f-623cc2b1ba76 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:22.799961 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3addd593-0b4c-4e7f-9b1f-623cc2b1ba76 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:22.800230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3addd593-0b4c-4e7f-9b1f-623cc2b1ba76 None admin] PATH_INFO: `/v3/users/24b1ed517c454625a2a733a06aa4fb70` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:22.801403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3addd593-0b4c-4e7f-9b1f-623cc2b1ba76 None admin] RBAC: Authorizing `identity:get_user(user_id=24b1ed517c454625a2a733a06aa4fb70)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:22.802957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3addd593-0b4c-4e7f-9b1f-623cc2b1ba76 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:22.804866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 110/224] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:22 2022] GET /identity/v3/users/24b1ed517c454625a2a733a06aa4fb70 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:22.805441 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3addd593-0b4c-4e7f-9b1f-623cc2b1ba76 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:22.813817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19510c-9bd8-4738-88c1-ed0aa5e1d6fa None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:22.849752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a6a0d-0e33-47ed-895e-b191ff429fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:22.850387 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a6a0d-0e33-47ed-895e-b191ff429fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:22.850667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a6a0d-0e33-47ed-895e-b191ff429fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:22.850956 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a6a0d-0e33-47ed-895e-b191ff429fe2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:22.863252 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a6a0d-0e33-47ed-895e-b191ff429fe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:22.868186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-660a6a0d-0e33-47ed-895e-b191ff429fe2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:22.868186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:22.868186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:22.868186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:22.868186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:22.868186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:22.868186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:22.868186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:22.868186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:22.868186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:22.868186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:22.868186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:22.868186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:22.868186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:22.868186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:22.868186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:22.869271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:22.869271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:22.869271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:22.869271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:22.869271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:22.869271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:22.869271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:22.869271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:22.869271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:22.869271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:22.869271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:22.869271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:22.869271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:22.869271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:22.869271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:22.869271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:22.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:22.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:22.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:22.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:22.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:22.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:22.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:22.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:22.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:22.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:22.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:22.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:22.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 115/225] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:53:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:22.871287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a6a0d-0e33-47ed-895e-b191ff429fe2 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:22.938929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3addd593-0b4c-4e7f-9b1f-623cc2b1ba76 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:22.977380 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626c53f-6780-48fb-834c-9ad37f8d9d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:22.978210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626c53f-6780-48fb-834c-9ad37f8d9d85 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:22.978487 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626c53f-6780-48fb-834c-9ad37f8d9d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:22.980476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626c53f-6780-48fb-834c-9ad37f8d9d85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:23.004099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a6a0d-0e33-47ed-895e-b191ff429fe2 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:23.029728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:23.040413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:23.049759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ec432-993a-450a-868b-2119445e1738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:23.050839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ec432-993a-450a-868b-2119445e1738 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:23.051194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ec432-993a-450a-868b-2119445e1738 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:23.051745 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ec432-993a-450a-868b-2119445e1738 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:23.053137 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ec432-993a-450a-868b-2119445e1738 None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:23.055223 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ec432-993a-450a-868b-2119445e1738 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:23.057668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 116/226] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:22 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:23.058689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ec432-993a-450a-868b-2119445e1738 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:23.066636 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626c53f-6780-48fb-834c-9ad37f8d9d85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:23.068152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626c53f-6780-48fb-834c-9ad37f8d9d85 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:23.181289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ec432-993a-450a-868b-2119445e1738 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:23.191073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 111/227] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:23.191899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626c53f-6780-48fb-834c-9ad37f8d9d85 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:23.225725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e73e1-6e44-45ce-a344-8b93dd724cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:23.226339 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e73e1-6e44-45ce-a344-8b93dd724cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:23.226613 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e73e1-6e44-45ce-a344-8b93dd724cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:23.226886 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e73e1-6e44-45ce-a344-8b93dd724cc6 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:23.228141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e73e1-6e44-45ce-a344-8b93dd724cc6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:23.228615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e73e1-6e44-45ce-a344-8b93dd724cc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:23.230084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e73e1-6e44-45ce-a344-8b93dd724cc6 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:23.239082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 117/228] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:53:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:23.239987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e73e1-6e44-45ce-a344-8b93dd724cc6 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:23.289338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:23.301506 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:23.332562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626c53f-6780-48fb-834c-9ad37f8d9d85 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:23.349665 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e73e1-6e44-45ce-a344-8b93dd724cc6 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:23.373971 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555fdb6-7636-4d75-8b4d-ccd9b474a618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:23.374891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555fdb6-7636-4d75-8b4d-ccd9b474a618 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:23.375576 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555fdb6-7636-4d75-8b4d-ccd9b474a618 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:23.375988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555fdb6-7636-4d75-8b4d-ccd9b474a618 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551/users/24b1ed517c454625a2a733a06aa4fb70/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:23.378761 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555fdb6-7636-4d75-8b4d-ccd9b474a618 None admin] RBAC: Authorizing `identity:create_grant(project_id=39c302b22dd04479bf8305109d613551, user_id=24b1ed517c454625a2a733a06aa4fb70, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:23.380444 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555fdb6-7636-4d75-8b4d-ccd9b474a618 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:23.387259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f9055-c56e-468e-9fdb-877e49a3785b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:23.387937 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f9055-c56e-468e-9fdb-877e49a3785b None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:23.388212 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f9055-c56e-468e-9fdb-877e49a3785b None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:23.388479 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f9055-c56e-468e-9fdb-877e49a3785b None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:23.389637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f9055-c56e-468e-9fdb-877e49a3785b None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:23.391270 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f9055-c56e-468e-9fdb-877e49a3785b None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:23.395079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 118/229] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:23 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:23.395079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f9055-c56e-468e-9fdb-877e49a3785b None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:23.447966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 112/230] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:53:23 2022] PUT /identity/v3/projects/39c302b22dd04479bf8305109d613551/users/24b1ed517c454625a2a733a06aa4fb70/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:53:23.448868 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555fdb6-7636-4d75-8b4d-ccd9b474a618 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:23.533677 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f9055-c56e-468e-9fdb-877e49a3785b None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:23.565454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555fdb6-7636-4d75-8b4d-ccd9b474a618 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:23.570714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:23.584178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fbd8a-ab7e-4033-8008-1db0036764f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:23.585445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fbd8a-ab7e-4033-8008-1db0036764f3 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:23.587457 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fbd8a-ab7e-4033-8008-1db0036764f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:23.587457 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fbd8a-ab7e-4033-8008-1db0036764f3 None admin] PATH_INFO: `/v3/users/24b1ed517c454625a2a733a06aa4fb70` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:23.588185 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fbd8a-ab7e-4033-8008-1db0036764f3 None admin] RBAC: Authorizing `identity:get_user(user_id=24b1ed517c454625a2a733a06aa4fb70)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:23.590486 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fbd8a-ab7e-4033-8008-1db0036764f3 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:23.593291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 119/231] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:23 2022] GET /identity/v3/users/24b1ed517c454625a2a733a06aa4fb70 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:23.599782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fbd8a-ab7e-4033-8008-1db0036764f3 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:23.604840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:23.612092 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbf53e-dab3-4dae-9210-819fde0c6897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:23.612874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbf53e-dab3-4dae-9210-819fde0c6897 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:23.613247 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbf53e-dab3-4dae-9210-819fde0c6897 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:23.613614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbf53e-dab3-4dae-9210-819fde0c6897 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:23.614757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbf53e-dab3-4dae-9210-819fde0c6897 None admin] RBAC: Adding query filter params (role.id=dd5cafa908a74cfd8e1a8baa44122242, scope.project.id=39c302b22dd04479bf8305109d613551, user.id=111855efac094907a56d3a29d5f61ee8) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:23.615315 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbf53e-dab3-4dae-9210-819fde0c6897 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:23.616893 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbf53e-dab3-4dae-9210-819fde0c6897 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:23.624030 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 113/232] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:23 2022] GET /identity/v3/role_assignments?user.id=111855efac094907a56d3a29d5f61ee8&scope.project.id=39c302b22dd04479bf8305109d613551&role.id=dd5cafa908a74cfd8e1a8baa44122242 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:23.733245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fbd8a-ab7e-4033-8008-1db0036764f3 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:23.772606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3849a2e-2d8b-426e-a0bf-53c415a77d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:23.773511 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3849a2e-2d8b-426e-a0bf-53c415a77d55 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:23.773929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3849a2e-2d8b-426e-a0bf-53c415a77d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:23.774422 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3849a2e-2d8b-426e-a0bf-53c415a77d55 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:23.775824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3849a2e-2d8b-426e-a0bf-53c415a77d55 None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:23.777448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3849a2e-2d8b-426e-a0bf-53c415a77d55 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:23.779347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 120/233] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:23 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:23.788375 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbf53e-dab3-4dae-9210-819fde0c6897 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:23.917073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbf53e-dab3-4dae-9210-819fde0c6897 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:23.960741 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbb40b-4ce6-4ba6-ac6f-7c4b0b863179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:23.961431 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbb40b-4ce6-4ba6-ac6f-7c4b0b863179 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:23.961712 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbb40b-4ce6-4ba6-ac6f-7c4b0b863179 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:23.962080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbb40b-4ce6-4ba6-ac6f-7c4b0b863179 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551/users/24b1ed517c454625a2a733a06aa4fb70/roles/d9b2237fd9884cc39771e75bf902e203` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:23.964887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbb40b-4ce6-4ba6-ac6f-7c4b0b863179 None admin] RBAC: Authorizing `identity:create_grant(project_id=39c302b22dd04479bf8305109d613551, user_id=24b1ed517c454625a2a733a06aa4fb70, role_id=d9b2237fd9884cc39771e75bf902e203)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:23.967377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbb40b-4ce6-4ba6-ac6f-7c4b0b863179 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:24.070983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 114/234] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:53:23 2022] PUT /identity/v3/projects/39c302b22dd04479bf8305109d613551/users/24b1ed517c454625a2a733a06aa4fb70/roles/d9b2237fd9884cc39771e75bf902e203 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:53:25.099603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57eecb-e447-42ef-9ef6-c06db8f031a9 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:25.100111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57eecb-e447-42ef-9ef6-c06db8f031a9 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:25.100214 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57eecb-e447-42ef-9ef6-c06db8f031a9 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:25.101733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 121/235] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:25.108130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58f192-93a2-4ba1-986c-241a6cb46e6a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:25.108580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58f192-93a2-4ba1-986c-241a6cb46e6a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:25.108660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58f192-93a2-4ba1-986c-241a6cb46e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:25.128147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ec58f192-93a2-4ba1-986c-241a6cb46e6a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:25.129332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ec58f192-93a2-4ba1-986c-241a6cb46e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:25.207695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:25.226886 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 115/236] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:25.461526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5794b-11bf-4969-b2c2-829fe5af72fa None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:25.462539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5794b-11bf-4969-b2c2-829fe5af72fa None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:25.462923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5794b-11bf-4969-b2c2-829fe5af72fa None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:25.464622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 122/237] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:25.470029 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7515a3-2f76-4259-b186-9fc9ab515f52 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:25.470416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7515a3-2f76-4259-b186-9fc9ab515f52 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:25.470787 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7515a3-2f76-4259-b186-9fc9ab515f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:25.493786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5f7515a3-2f76-4259-b186-9fc9ab515f52 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:25.494968 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5f7515a3-2f76-4259-b186-9fc9ab515f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:25.554557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5794b-11bf-4969-b2c2-829fe5af72fa None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:25.578961 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 116/238] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:25.640667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:25.684386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5794b-11bf-4969-b2c2-829fe5af72fa None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:25.729830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e8c93-52c0-42dc-a09b-71e43c29198b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:25.730380 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e8c93-52c0-42dc-a09b-71e43c29198b None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:25.730653 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e8c93-52c0-42dc-a09b-71e43c29198b None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:25.730922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e8c93-52c0-42dc-a09b-71e43c29198b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:25.746154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e8c93-52c0-42dc-a09b-71e43c29198b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:25.751688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-a14e8c93-52c0-42dc-a09b-71e43c29198b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:25.751688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:25.751688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:25.751688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:25.751688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:25.751688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:25.751688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:25.751688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:25.751688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:25.751688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:25.751688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:25.751688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:25.751688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:25.751688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:25.751688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:25.751688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:25.752597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:25.752597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:25.752597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:25.752597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:25.752597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:25.752597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:25.752597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:25.752597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:25.752597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:25.752597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:25.752597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:25.752597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:25.752597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:25.752597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:25.752597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:25.752597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:25.753368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:25.753368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:25.753368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:25.753368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:25.753368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:25.753368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:25.753368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:25.753368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:25.753368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:25.753368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:25.753368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:25.753368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:25.753368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 123/239] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:53:25.763902 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7515a3-2f76-4259-b186-9fc9ab515f52 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:25.898266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7515a3-2f76-4259-b186-9fc9ab515f52 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:25.919772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:25.944036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5d5c7-19e1-4ebe-824a-4330b7537d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:25.944659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d5c7-19e1-4ebe-824a-4330b7537d4c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:25.944934 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d5c7-19e1-4ebe-824a-4330b7537d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:25.945211 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d5c7-19e1-4ebe-824a-4330b7537d4c None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:25.945887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5d5c7-19e1-4ebe-824a-4330b7537d4c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:25.947410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5d5c7-19e1-4ebe-824a-4330b7537d4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:25.948816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5d5c7-19e1-4ebe-824a-4330b7537d4c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:25.967229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 117/240] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:25.969890 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e8c93-52c0-42dc-a09b-71e43c29198b None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:26.010741 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5d5c7-19e1-4ebe-824a-4330b7537d4c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:26.191782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5d5c7-19e1-4ebe-824a-4330b7537d4c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:26.234659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781fcf2-75e3-4384-8a29-e08d876d2f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:26.236850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e8c93-52c0-42dc-a09b-71e43c29198b None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:26.237964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781fcf2-75e3-4384-8a29-e08d876d2f19 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:26.238389 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781fcf2-75e3-4384-8a29-e08d876d2f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:26.238803 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781fcf2-75e3-4384-8a29-e08d876d2f19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:26.241606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781fcf2-75e3-4384-8a29-e08d876d2f19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:26.243649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781fcf2-75e3-4384-8a29-e08d876d2f19 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:26.252603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 118/241] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:53:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:26.257838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781fcf2-75e3-4384-8a29-e08d876d2f19 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:26.300512 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:26.312414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:26.314677 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8216243-656b-403d-b388-668b0bfdc505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:26.315524 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8216243-656b-403d-b388-668b0bfdc505 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:26.315697 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8216243-656b-403d-b388-668b0bfdc505 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:26.315969 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8216243-656b-403d-b388-668b0bfdc505 None admin] PATH_INFO: `/v3/users/39a48bea33ef4fc3a474aef8cdecdcc6` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:26.332839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8216243-656b-403d-b388-668b0bfdc505 None admin] RBAC: Authorizing `identity:get_user(user_id=39a48bea33ef4fc3a474aef8cdecdcc6)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:26.334680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8216243-656b-403d-b388-668b0bfdc505 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:26.337693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 124/242] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:25 2022] GET /identity/v3/users/39a48bea33ef4fc3a474aef8cdecdcc6 => generated 297 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:26.339237 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05807c7-9602-4337-8b9d-956746a8945f None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:26.339710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05807c7-9602-4337-8b9d-956746a8945f None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:26.340155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05807c7-9602-4337-8b9d-956746a8945f None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:26.341622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 125/243] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:26.346993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05807c7-9602-4337-8b9d-956746a8945f None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:26.385231 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781fcf2-75e3-4384-8a29-e08d876d2f19 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:26.429319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f4ce5-157f-460b-9efa-aa6cd8460801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:26.429934 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f4ce5-157f-460b-9efa-aa6cd8460801 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:26.430213 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f4ce5-157f-460b-9efa-aa6cd8460801 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:26.430515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f4ce5-157f-460b-9efa-aa6cd8460801 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:26.431212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f4ce5-157f-460b-9efa-aa6cd8460801 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:26.432663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f4ce5-157f-460b-9efa-aa6cd8460801 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:26.460456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05807c7-9602-4337-8b9d-956746a8945f None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:26.495136 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eccc2c-25ef-4ee6-b48b-392f85b20022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:26.495747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eccc2c-25ef-4ee6-b48b-392f85b20022 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:26.496026 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eccc2c-25ef-4ee6-b48b-392f85b20022 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:26.496297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eccc2c-25ef-4ee6-b48b-392f85b20022 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:26.511055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 119/244] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:53:26 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:53:26.512723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7df71a-41b5-409c-b1bc-4099edc7dfc1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:26.513396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7df71a-41b5-409c-b1bc-4099edc7dfc1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:26.513751 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7df71a-41b5-409c-b1bc-4099edc7dfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:26.534716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2d7df71a-41b5-409c-b1bc-4099edc7dfc1 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:26.536086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2d7df71a-41b5-409c-b1bc-4099edc7dfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:26.571612 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:26.592931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eccc2c-25ef-4ee6-b48b-392f85b20022 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:26.595183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eccc2c-25ef-4ee6-b48b-392f85b20022 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:26.625156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 120/245] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:26.625156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff119ed1-5cba-4922-8e48-35e5fd12016d None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:26.625156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff119ed1-5cba-4922-8e48-35e5fd12016d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:26.625156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff119ed1-5cba-4922-8e48-35e5fd12016d None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:26.625156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 121/246] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:26.628158 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff829e7-4ade-43c1-bfdd-61794a3940bc None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:26.630067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff829e7-4ade-43c1-bfdd-61794a3940bc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:26.630548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff829e7-4ade-43c1-bfdd-61794a3940bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:26.669105 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8ff829e7-4ade-43c1-bfdd-61794a3940bc None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:26.674786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8ff829e7-4ade-43c1-bfdd-61794a3940bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:26.720532 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:26.772617 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 122/247] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:26.775015 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 126/248] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:26.782993 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff829e7-4ade-43c1-bfdd-61794a3940bc None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:26.892398 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff829e7-4ade-43c1-bfdd-61794a3940bc None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:26.915875 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eccc2c-25ef-4ee6-b48b-392f85b20022 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:26.933145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9002f-0abc-4fa2-bd95-328dae44a6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:26.933838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9002f-0abc-4fa2-bd95-328dae44a6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:26.934106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9002f-0abc-4fa2-bd95-328dae44a6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:26.934491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9002f-0abc-4fa2-bd95-328dae44a6a0 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:26.935976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9002f-0abc-4fa2-bd95-328dae44a6a0 None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:26.937928 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9002f-0abc-4fa2-bd95-328dae44a6a0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:26.940139 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 123/249] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:26 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:26.941506 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ca206-fdcf-41c2-94ab-10044d4273c2 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:26.941783 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ca206-fdcf-41c2-94ab-10044d4273c2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:26.942379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ca206-fdcf-41c2-94ab-10044d4273c2 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:26.943853 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 124/250] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:26.945308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ca206-fdcf-41c2-94ab-10044d4273c2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:26.981981 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:27.016418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:27.038405 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eccc2c-25ef-4ee6-b48b-392f85b20022 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:27.077877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ca206-fdcf-41c2-94ab-10044d4273c2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:27.089997 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3366637-656a-4c12-81a6-c2fa697798f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:27.091181 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3366637-656a-4c12-81a6-c2fa697798f5 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:27.091709 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3366637-656a-4c12-81a6-c2fa697798f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:27.092187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3366637-656a-4c12-81a6-c2fa697798f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:27.108169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3366637-656a-4c12-81a6-c2fa697798f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:27.114549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-f3366637-656a-4c12-81a6-c2fa697798f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:27.114549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:27.114549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:27.114549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:27.114549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:27.114549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:27.114549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:27.114549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:27.114549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:27.114549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:27.114549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:27.114549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:27.114549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:27.114549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:27.114549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:27.114549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:27.115640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:27.115640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:27.115640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:27.115640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:27.115640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:27.115640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:27.115640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:27.115640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:27.115640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:27.115640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:27.115640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:27.115640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:27.115640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:27.115640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:27.115640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:27.115640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:27.116598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:27.116598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:27.116598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:27.116598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:27.116598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:27.116598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:27.116598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:27.116598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:27.116598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:27.116598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:27.116598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:27.116598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:27.119427 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 127/251] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:53:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:27.120520 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60834f65-c395-495f-88f7-153aa5715ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:27.121017 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f93c0-f2e2-4e57-b16b-7b77d4fe9fd0 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:27.121187 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60834f65-c395-495f-88f7-153aa5715ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:27.121459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60834f65-c395-495f-88f7-153aa5715ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:27.121582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f93c0-f2e2-4e57-b16b-7b77d4fe9fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:27.121758 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60834f65-c395-495f-88f7-153aa5715ce2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:27.121850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f93c0-f2e2-4e57-b16b-7b77d4fe9fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:27.122443 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60834f65-c395-495f-88f7-153aa5715ce2 None admin] RBAC: Adding query filter params (role.id=74955b19ba90422cbe29fa461c3f287f, scope.project.id=39c302b22dd04479bf8305109d613551, user.id=39a48bea33ef4fc3a474aef8cdecdcc6) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:27.122745 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60834f65-c395-495f-88f7-153aa5715ce2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:27.124522 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60834f65-c395-495f-88f7-153aa5715ce2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:27.130635 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 125/252] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:26 2022] GET /identity/v3/role_assignments?user.id=39a48bea33ef4fc3a474aef8cdecdcc6&scope.project.id=39c302b22dd04479bf8305109d613551&role.id=74955b19ba90422cbe29fa461c3f287f => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:27.131511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60834f65-c395-495f-88f7-153aa5715ce2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:27.167568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-c26f93c0-f2e2-4e57-b16b-7b77d4fe9fd0 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:27.169025 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-c26f93c0-f2e2-4e57-b16b-7b77d4fe9fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:27.251746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:27.263951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 128/253] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:27.266701 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f93c0-f2e2-4e57-b16b-7b77d4fe9fd0 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:27.280400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60834f65-c395-495f-88f7-153aa5715ce2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:27.377192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33e5f0-3e13-419a-acaa-d293b4f8e199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:27.378154 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33e5f0-3e13-419a-acaa-d293b4f8e199 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:27.378565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33e5f0-3e13-419a-acaa-d293b4f8e199 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:27.378990 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33e5f0-3e13-419a-acaa-d293b4f8e199 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:27.406239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da33e5f0-3e13-419a-acaa-d293b4f8e199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:27.412508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-da33e5f0-3e13-419a-acaa-d293b4f8e199 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:27.412508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:27.412508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:27.412508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:27.412508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:27.412508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:27.412508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:27.412508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:27.412508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:27.412508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:27.412508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:27.412508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:27.412508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:27.412508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:27.412508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:27.412508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:27.413466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:27.413466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:27.413466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:27.413466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:27.413466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:27.413466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:27.413466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:27.413466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:27.413466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:27.413466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:27.413466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:27.413466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:27.413466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:27.413466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:27.413466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:27.413466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:27.414287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:27.414287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:27.414287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:27.414287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:27.414287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:27.414287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:27.414287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:27.414287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:27.414287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:27.414287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:27.414287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:27.414287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:27.414287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 126/254] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:27.422455 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33e5f0-3e13-419a-acaa-d293b4f8e199 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:27.498300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:27.525259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f93c0-f2e2-4e57-b16b-7b77d4fe9fd0 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:27.542363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33e5f0-3e13-419a-acaa-d293b4f8e199 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:27.580720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f932f37-6e33-496a-b94b-40ba46e037a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:27.581359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f932f37-6e33-496a-b94b-40ba46e037a5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:27.581637 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f932f37-6e33-496a-b94b-40ba46e037a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:27.581905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f932f37-6e33-496a-b94b-40ba46e037a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:27.582575 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f932f37-6e33-496a-b94b-40ba46e037a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:27.582876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f932f37-6e33-496a-b94b-40ba46e037a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:27.584277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f932f37-6e33-496a-b94b-40ba46e037a5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:27.594836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 127/255] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:27.600404 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f932f37-6e33-496a-b94b-40ba46e037a5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:27.606384 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe3e30-9cb7-448a-b2ea-bc28a58a32e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:27.607030 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe3e30-9cb7-448a-b2ea-bc28a58a32e5 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:27.607305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe3e30-9cb7-448a-b2ea-bc28a58a32e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:27.607572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe3e30-9cb7-448a-b2ea-bc28a58a32e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:27.608246 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe3e30-9cb7-448a-b2ea-bc28a58a32e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:27.608543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe3e30-9cb7-448a-b2ea-bc28a58a32e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:27.609926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe3e30-9cb7-448a-b2ea-bc28a58a32e5 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:27.619148 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:27.626143 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 129/256] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:27.636002 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe3e30-9cb7-448a-b2ea-bc28a58a32e5 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:27.737862 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f932f37-6e33-496a-b94b-40ba46e037a5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:27.782116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443cbd3-8f00-4033-a756-b31f76b36734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:27.783206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443cbd3-8f00-4033-a756-b31f76b36734 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:27.783666 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443cbd3-8f00-4033-a756-b31f76b36734 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:27.784115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443cbd3-8f00-4033-a756-b31f76b36734 None admin] PATH_INFO: `/v3/users/111855efac094907a56d3a29d5f61ee8` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:27.786269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443cbd3-8f00-4033-a756-b31f76b36734 None admin] RBAC: Authorizing `identity:get_user(user_id=111855efac094907a56d3a29d5f61ee8)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:27.788837 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443cbd3-8f00-4033-a756-b31f76b36734 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:27.793044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 128/257] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:27 2022] GET /identity/v3/users/111855efac094907a56d3a29d5f61ee8 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:27.793875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443cbd3-8f00-4033-a756-b31f76b36734 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:27.877706 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe3e30-9cb7-448a-b2ea-bc28a58a32e5 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:27.892997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:27.913478 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443cbd3-8f00-4033-a756-b31f76b36734 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:27.951822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:27.956210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33a34b-faac-484a-8e7f-2972c9de0cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:27.957207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33a34b-faac-484a-8e7f-2972c9de0cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:27.957821 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33a34b-faac-484a-8e7f-2972c9de0cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:27.958267 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33a34b-faac-484a-8e7f-2972c9de0cc5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:27.962871 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2098fc-64bd-45cb-ba28-7e4a7fc9f6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:27.966511 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2098fc-64bd-45cb-ba28-7e4a7fc9f6ca None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:27.966511 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2098fc-64bd-45cb-ba28-7e4a7fc9f6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:27.966511 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2098fc-64bd-45cb-ba28-7e4a7fc9f6ca None admin] PATH_INFO: `/v3/users/24b1ed517c454625a2a733a06aa4fb70` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:27.967310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2098fc-64bd-45cb-ba28-7e4a7fc9f6ca None admin] RBAC: Authorizing `identity:get_user(user_id=24b1ed517c454625a2a733a06aa4fb70)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:27.968873 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2098fc-64bd-45cb-ba28-7e4a7fc9f6ca None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:27.970653 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 130/258] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:27 2022] GET /identity/v3/users/24b1ed517c454625a2a733a06aa4fb70 => generated 283 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:27.971344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33a34b-faac-484a-8e7f-2972c9de0cc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:27.973977 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2098fc-64bd-45cb-ba28-7e4a7fc9f6ca None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:27.976693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6d33a34b-faac-484a-8e7f-2972c9de0cc5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:27.976693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:27.976693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:27.976693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:27.976693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:27.976693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:27.976693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:27.976693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:27.976693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:27.976693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:27.976693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:27.976693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:27.976693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:27.976693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:27.976693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:27.976693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:27.977620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:27.977620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:27.977620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:27.977620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:27.977620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:27.977620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:27.977620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:27.977620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:27.977620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:27.977620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:27.977620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:27.977620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:27.977620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:27.977620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:27.977620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:27.977620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:27.978718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:27.978718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:27.978718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:27.978718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:27.978718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:27.978718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:27.978718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:27.978718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:27.978718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:27.978718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:27.978718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:27.978718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:27.980736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 129/259] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:53:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:27.981677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33a34b-faac-484a-8e7f-2972c9de0cc5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:28.101862 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33a34b-faac-484a-8e7f-2972c9de0cc5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:28.143980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf1145-e521-4324-be0d-784fbd30e336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:28.144999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf1145-e521-4324-be0d-784fbd30e336 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:28.145467 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf1145-e521-4324-be0d-784fbd30e336 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:28.146013 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf1145-e521-4324-be0d-784fbd30e336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:28.164954 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:28.199894 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2098fc-64bd-45cb-ba28-7e4a7fc9f6ca None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:28.263993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d2474-88ea-485d-9b56-7dbf291c2b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:28.265436 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d2474-88ea-485d-9b56-7dbf291c2b65 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:28.267156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d2474-88ea-485d-9b56-7dbf291c2b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:28.267689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d2474-88ea-485d-9b56-7dbf291c2b65 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:28.270208 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d2474-88ea-485d-9b56-7dbf291c2b65 None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:28.272734 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d2474-88ea-485d-9b56-7dbf291c2b65 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:28.277060 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 131/260] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:27 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:28.286704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d2474-88ea-485d-9b56-7dbf291c2b65 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:28.295066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf1145-e521-4324-be0d-784fbd30e336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:28.297153 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf1145-e521-4324-be0d-784fbd30e336 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:28.419164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 130/261] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:28.420370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf1145-e521-4324-be0d-784fbd30e336 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:28.420831 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:28.468489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:28.509264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d2474-88ea-485d-9b56-7dbf291c2b65 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:28.532959 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf1145-e521-4324-be0d-784fbd30e336 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:28.572113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efbd414-b314-4aae-b5dc-0733f1c2f743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:28.573105 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbd414-b314-4aae-b5dc-0733f1c2f743 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:28.573574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbd414-b314-4aae-b5dc-0733f1c2f743 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:28.574053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbd414-b314-4aae-b5dc-0733f1c2f743 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551/users/111855efac094907a56d3a29d5f61ee8/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:28.576501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efbd414-b314-4aae-b5dc-0733f1c2f743 None admin] RBAC: Authorizing `identity:create_grant(project_id=39c302b22dd04479bf8305109d613551, user_id=111855efac094907a56d3a29d5f61ee8, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:28.578162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efbd414-b314-4aae-b5dc-0733f1c2f743 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:28.594274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042ebde-a421-4552-b17b-2f045793ccb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:28.594891 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042ebde-a421-4552-b17b-2f045793ccb5 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:28.595199 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042ebde-a421-4552-b17b-2f045793ccb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:28.600988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042ebde-a421-4552-b17b-2f045793ccb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:28.600988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042ebde-a421-4552-b17b-2f045793ccb5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:28.600988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042ebde-a421-4552-b17b-2f045793ccb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:28.600988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042ebde-a421-4552-b17b-2f045793ccb5 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:28.606805 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 132/262] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:53:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:28.607642 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042ebde-a421-4552-b17b-2f045793ccb5 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:28.663981 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 131/263] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:53:28 2022] PUT /identity/v3/projects/39c302b22dd04479bf8305109d613551/users/111855efac094907a56d3a29d5f61ee8/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:53:28.666633 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efbd414-b314-4aae-b5dc-0733f1c2f743 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:28.775052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042ebde-a421-4552-b17b-2f045793ccb5 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:28.835083 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:28.849790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084bcf4-524e-442f-9b98-606686b4cfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:28.850448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084bcf4-524e-442f-9b98-606686b4cfd8 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:28.850729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084bcf4-524e-442f-9b98-606686b4cfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:28.851032 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084bcf4-524e-442f-9b98-606686b4cfd8 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:28.852962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084bcf4-524e-442f-9b98-606686b4cfd8 None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:28.856030 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084bcf4-524e-442f-9b98-606686b4cfd8 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:28.859078 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 133/264] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:28 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:28.864354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084bcf4-524e-442f-9b98-606686b4cfd8 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:28.867195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efbd414-b314-4aae-b5dc-0733f1c2f743 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:28.869364 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:28.942058 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392141bc-16f0-46d6-9006-215de9995c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:28.942706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392141bc-16f0-46d6-9006-215de9995c18 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:28.943009 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392141bc-16f0-46d6-9006-215de9995c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:28.943287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392141bc-16f0-46d6-9006-215de9995c18 None admin] PATH_INFO: `/v3/users/24b1ed517c454625a2a733a06aa4fb70` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:28.944494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392141bc-16f0-46d6-9006-215de9995c18 None admin] RBAC: Authorizing `identity:get_user(user_id=24b1ed517c454625a2a733a06aa4fb70)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:28.946535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392141bc-16f0-46d6-9006-215de9995c18 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:28.948492 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 132/265] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:28 2022] GET /identity/v3/users/24b1ed517c454625a2a733a06aa4fb70 => generated 283 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:28.953684 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392141bc-16f0-46d6-9006-215de9995c18 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:29.030804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084bcf4-524e-442f-9b98-606686b4cfd8 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:29.083978 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c553d2-b5bb-46cf-9357-8463f2f53566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:29.084623 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c553d2-b5bb-46cf-9357-8463f2f53566 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:29.084910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c553d2-b5bb-46cf-9357-8463f2f53566 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:29.085192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c553d2-b5bb-46cf-9357-8463f2f53566 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:29.087287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c553d2-b5bb-46cf-9357-8463f2f53566 None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=39c302b22dd04479bf8305109d613551, user.id=24b1ed517c454625a2a733a06aa4fb70) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:29.087620 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c553d2-b5bb-46cf-9357-8463f2f53566 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:29.090963 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c553d2-b5bb-46cf-9357-8463f2f53566 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:29.097123 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 134/266] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:28 2022] GET /identity/v3/role_assignments?user.id=24b1ed517c454625a2a733a06aa4fb70&scope.project.id=39c302b22dd04479bf8305109d613551&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:29.161138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392141bc-16f0-46d6-9006-215de9995c18 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:29.211095 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6aa81-ec01-4ccb-a3cb-60367cd222e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:29.212597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6aa81-ec01-4ccb-a3cb-60367cd222e2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:29.213018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6aa81-ec01-4ccb-a3cb-60367cd222e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:29.213423 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6aa81-ec01-4ccb-a3cb-60367cd222e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:29.269387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:29.308056 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6aa81-ec01-4ccb-a3cb-60367cd222e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:29.309474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6aa81-ec01-4ccb-a3cb-60367cd222e2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:29.459183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 133/267] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 07:53:29.466210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c553d2-b5bb-46cf-9357-8463f2f53566 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:29.549217 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:29.572199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf05fa3-d9e4-4788-be54-e001dd8175ca None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:29.572377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf05fa3-d9e4-4788-be54-e001dd8175ca None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:29.572653 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf05fa3-d9e4-4788-be54-e001dd8175ca None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:29.579040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 134/268] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:29.586363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75365b47-ae6f-4fe6-abcc-6ea97396a49c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:29.586797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75365b47-ae6f-4fe6-abcc-6ea97396a49c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:29.587159 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75365b47-ae6f-4fe6-abcc-6ea97396a49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:29.593023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c553d2-b5bb-46cf-9357-8463f2f53566 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:29.612245 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-75365b47-ae6f-4fe6-abcc-6ea97396a49c None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:29.613458 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-75365b47-ae6f-4fe6-abcc-6ea97396a49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:29.651344 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09341f07-d4b7-4f99-a379-5b71351124ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:29.652340 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09341f07-d4b7-4f99-a379-5b71351124ee None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:29.653041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09341f07-d4b7-4f99-a379-5b71351124ee None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:29.653525 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09341f07-d4b7-4f99-a379-5b71351124ee None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:29.656657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09341f07-d4b7-4f99-a379-5b71351124ee None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:29.658501 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09341f07-d4b7-4f99-a379-5b71351124ee None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:29.660428 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 135/269] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:29 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:29.672517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a2590-5f33-4c27-8eb6-942c1af72c5f None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:29.673378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a2590-5f33-4c27-8eb6-942c1af72c5f None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:29.674278 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a2590-5f33-4c27-8eb6-942c1af72c5f None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:29.682980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 136/270] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:29.684394 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a2590-5f33-4c27-8eb6-942c1af72c5f None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:29.729106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:29.737410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 135/271] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:29.739123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330008c7-1fdb-413c-a536-379344a17e8c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:29.739588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330008c7-1fdb-413c-a536-379344a17e8c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:29.740021 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330008c7-1fdb-413c-a536-379344a17e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:29.779589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-330008c7-1fdb-413c-a536-379344a17e8c None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:29.781112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-330008c7-1fdb-413c-a536-379344a17e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:29.812419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a2590-5f33-4c27-8eb6-942c1af72c5f None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:29.852688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:29.862753 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ef973-1894-4ee6-9082-60ba3867191d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:29.863388 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ef973-1894-4ee6-9082-60ba3867191d None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:29.870396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ef973-1894-4ee6-9082-60ba3867191d None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:29.870396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ef973-1894-4ee6-9082-60ba3867191d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:29.870396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ef973-1894-4ee6-9082-60ba3867191d None admin] RBAC: Adding query filter params (role.id=d9b2237fd9884cc39771e75bf902e203, scope.project.id=39c302b22dd04479bf8305109d613551, user.id=24b1ed517c454625a2a733a06aa4fb70) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:29.870396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ef973-1894-4ee6-9082-60ba3867191d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:29.870396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ef973-1894-4ee6-9082-60ba3867191d None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:29.873668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 137/272] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:29 2022] GET /identity/v3/role_assignments?user.id=24b1ed517c454625a2a733a06aa4fb70&scope.project.id=39c302b22dd04479bf8305109d613551&role.id=d9b2237fd9884cc39771e75bf902e203 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:29.917721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 136/273] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:30.045717 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ef973-1894-4ee6-9082-60ba3867191d None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:30.213724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ef973-1894-4ee6-9082-60ba3867191d None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:30.251782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1781815-a55b-44ed-8dd6-f096e7be66a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:30.252782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1781815-a55b-44ed-8dd6-f096e7be66a4 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:30.253254 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1781815-a55b-44ed-8dd6-f096e7be66a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:30.253722 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1781815-a55b-44ed-8dd6-f096e7be66a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:30.255104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1781815-a55b-44ed-8dd6-f096e7be66a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:30.256765 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1781815-a55b-44ed-8dd6-f096e7be66a4 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:30.258852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 138/274] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:53:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:30.265094 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330008c7-1fdb-413c-a536-379344a17e8c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:30.300670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1781815-a55b-44ed-8dd6-f096e7be66a4 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:30.391937 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:30.394849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330008c7-1fdb-413c-a536-379344a17e8c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:30.435581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:30.443260 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11c160-988e-4ae2-a646-63057075dca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:30.443983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11c160-988e-4ae2-a646-63057075dca6 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:30.444265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11c160-988e-4ae2-a646-63057075dca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:30.444536 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11c160-988e-4ae2-a646-63057075dca6 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:30.447806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11c160-988e-4ae2-a646-63057075dca6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:30.447806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11c160-988e-4ae2-a646-63057075dca6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:30.457197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1781815-a55b-44ed-8dd6-f096e7be66a4 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:30.494005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd834e8-529e-4599-a9cb-0e6d2efeeff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:30.494645 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd834e8-529e-4599-a9cb-0e6d2efeeff0 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:30.494901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd834e8-529e-4599-a9cb-0e6d2efeeff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:30.495206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd834e8-529e-4599-a9cb-0e6d2efeeff0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:30.507446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd834e8-529e-4599-a9cb-0e6d2efeeff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:30.513361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-acd834e8-529e-4599-a9cb-0e6d2efeeff0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:30.513361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:30.513361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:30.513361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:30.513361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:30.513361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:30.513361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:30.513361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:30.513361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:30.513361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:30.513361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:30.513361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:30.513361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:30.513361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:30.513361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:30.513361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:30.514495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:30.514495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:30.514495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:30.514495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:30.514495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:30.514495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:30.514495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:30.514495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:30.514495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:30.514495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:30.514495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:30.514495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:30.514495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:30.514495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:30.514495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:30.514495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:30.515594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:30.515594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:30.515594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:30.515594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:30.515594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:30.515594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:30.515594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:30.515594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:30.515594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:30.515594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:30.515594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:30.515594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:30.515594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 139/275] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:30.516403 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e229a-41f0-4e87-ba9a-9de9db2d226c None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:30.516403 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e229a-41f0-4e87-ba9a-9de9db2d226c None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:30.516403 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e229a-41f0-4e87-ba9a-9de9db2d226c None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:30.517579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 140/276] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:30.525070 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52679d92-40d7-46fd-b8a0-78956d0c6675 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:30.525070 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52679d92-40d7-46fd-b8a0-78956d0c6675 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:30.525070 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52679d92-40d7-46fd-b8a0-78956d0c6675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:30.541871 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-52679d92-40d7-46fd-b8a0-78956d0c6675 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:30.543073 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-52679d92-40d7-46fd-b8a0-78956d0c6675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:30.560812 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 137/277] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:53:30 2022] POST /identity/v3/users => generated 293 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:53:30.561640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11c160-988e-4ae2-a646-63057075dca6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:30.630419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 141/278] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:30.698048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11c160-988e-4ae2-a646-63057075dca6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:30.736027 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534501b-dbb2-4d03-9461-8fcc67459714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:30.737002 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534501b-dbb2-4d03-9461-8fcc67459714 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:30.737503 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534501b-dbb2-4d03-9461-8fcc67459714 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:30.737974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534501b-dbb2-4d03-9461-8fcc67459714 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:30.738853 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534501b-dbb2-4d03-9461-8fcc67459714 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:30.739395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534501b-dbb2-4d03-9461-8fcc67459714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:30.741708 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534501b-dbb2-4d03-9461-8fcc67459714 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:30.752245 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 138/279] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:30.757974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52679d92-40d7-46fd-b8a0-78956d0c6675 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:30.817529 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:30.867269 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52679d92-40d7-46fd-b8a0-78956d0c6675 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:30.905597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8e3b0-5cf8-48fa-a718-b082a4a5745b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:30.906236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8e3b0-5cf8-48fa-a718-b082a4a5745b None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:30.906510 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8e3b0-5cf8-48fa-a718-b082a4a5745b None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:30.906778 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8e3b0-5cf8-48fa-a718-b082a4a5745b None admin] PATH_INFO: `/v3/users/76106b84b2564b359b48c9c9fe5993a2` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:30.921979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534501b-dbb2-4d03-9461-8fcc67459714 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:30.922800 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8e3b0-5cf8-48fa-a718-b082a4a5745b None admin] RBAC: Authorizing `identity:get_user(user_id=76106b84b2564b359b48c9c9fe5993a2)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:30.924626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8e3b0-5cf8-48fa-a718-b082a4a5745b None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:30.926519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 142/280] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:30 2022] GET /identity/v3/users/76106b84b2564b359b48c9c9fe5993a2 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:30.934918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8e3b0-5cf8-48fa-a718-b082a4a5745b None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:30.994262 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:31.056508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534501b-dbb2-4d03-9461-8fcc67459714 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:31.058335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8e3b0-5cf8-48fa-a718-b082a4a5745b None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:31.072887 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:31.099304 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cc101-7a59-4aa4-9fe8-c348e12dcca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:31.100182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cc101-7a59-4aa4-9fe8-c348e12dcca0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:31.100862 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cc101-7a59-4aa4-9fe8-c348e12dcca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:31.101279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cc101-7a59-4aa4-9fe8-c348e12dcca0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:31.105805 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23b3b1-5420-4f75-9b3c-a38748b5e004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:31.106503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23b3b1-5420-4f75-9b3c-a38748b5e004 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:31.106997 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23b3b1-5420-4f75-9b3c-a38748b5e004 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:31.107449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23b3b1-5420-4f75-9b3c-a38748b5e004 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:31.108200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23b3b1-5420-4f75-9b3c-a38748b5e004 None admin] RBAC: Adding query filter params (role.id=dd5cafa908a74cfd8e1a8baa44122242, user.id=76106b84b2564b359b48c9c9fe5993a2, scope.system=all) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:31.108509 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23b3b1-5420-4f75-9b3c-a38748b5e004 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:31.112144 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23b3b1-5420-4f75-9b3c-a38748b5e004 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:31.120092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 143/281] 10.222.0.30 () {56 vars in 1308 bytes} [Thu Oct 6 07:53:30 2022] GET /identity/v3/role_assignments?user.id=76106b84b2564b359b48c9c9fe5993a2&scope.system=all&role.id=dd5cafa908a74cfd8e1a8baa44122242 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:31.126973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cc101-7a59-4aa4-9fe8-c348e12dcca0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:31.129848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-b72cc101-7a59-4aa4-9fe8-c348e12dcca0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:31.129848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:31.129848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:31.129848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:31.129848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:31.129848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:31.129848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:31.129848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:31.129848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:31.129848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:31.129848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:31.129848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:31.129848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:31.129848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:31.129848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:31.129848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:31.130754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:31.130754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:31.130754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:31.130754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:31.130754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:31.130754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:31.130754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:31.130754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:31.130754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:31.130754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:31.130754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:31.130754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:31.130754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:31.130754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:31.130754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:31.130754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:31.131746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:31.131746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:31.131746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:31.131746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:31.131746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:31.131746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:31.131746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:31.131746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:31.131746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:31.131746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:31.131746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:31.131746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:31.131746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 139/282] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:53:31.142418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23b3b1-5420-4f75-9b3c-a38748b5e004 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:31.310960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23b3b1-5420-4f75-9b3c-a38748b5e004 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:31.362603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4192763-06d1-4be6-a194-c88513e06358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:31.363631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4192763-06d1-4be6-a194-c88513e06358 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:31.364116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4192763-06d1-4be6-a194-c88513e06358 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:31.364616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4192763-06d1-4be6-a194-c88513e06358 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:31.365507 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4192763-06d1-4be6-a194-c88513e06358 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:31.366043 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4192763-06d1-4be6-a194-c88513e06358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:31.367782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4192763-06d1-4be6-a194-c88513e06358 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:31.378740 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 144/283] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:31.386974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cc101-7a59-4aa4-9fe8-c348e12dcca0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:31.532326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cc101-7a59-4aa4-9fe8-c348e12dcca0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:31.546419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64ab97-275e-4bbb-896b-2a1e11c9c632 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:31.547365 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64ab97-275e-4bbb-896b-2a1e11c9c632 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:31.547984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64ab97-275e-4bbb-896b-2a1e11c9c632 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:31.550282 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 145/284] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:53:31.556482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebea22-f13b-4c62-b300-0f835a0cf626 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:31.557193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebea22-f13b-4c62-b300-0f835a0cf626 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:31.557673 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebea22-f13b-4c62-b300-0f835a0cf626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:31.559319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:31.578910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-3eebea22-f13b-4c62-b300-0f835a0cf626 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:31.580515 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3eebea22-f13b-4c62-b300-0f835a0cf626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:31.585141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa5f3b-7ba3-4546-9679-0e3a4819d2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:31.585780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa5f3b-7ba3-4546-9679-0e3a4819d2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:31.586057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa5f3b-7ba3-4546-9679-0e3a4819d2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:31.586326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa5f3b-7ba3-4546-9679-0e3a4819d2b0 None admin] PATH_INFO: `/v3/users/39a48bea33ef4fc3a474aef8cdecdcc6` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:31.591701 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa5f3b-7ba3-4546-9679-0e3a4819d2b0 None admin] RBAC: Authorizing `identity:get_user(user_id=39a48bea33ef4fc3a474aef8cdecdcc6)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:31.593164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa5f3b-7ba3-4546-9679-0e3a4819d2b0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:31.594806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 140/285] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:31 2022] GET /identity/v3/users/39a48bea33ef4fc3a474aef8cdecdcc6 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:31.600410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa5f3b-7ba3-4546-9679-0e3a4819d2b0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:31.625900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:31.708066 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 146/286] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:31.727123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa5f3b-7ba3-4546-9679-0e3a4819d2b0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:31.770478 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef5024-3e38-4c43-81b0-b38efc4461b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:31.771377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef5024-3e38-4c43-81b0-b38efc4461b8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:31.771796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef5024-3e38-4c43-81b0-b38efc4461b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:31.772204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef5024-3e38-4c43-81b0-b38efc4461b8 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:31.773476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef5024-3e38-4c43-81b0-b38efc4461b8 None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:31.775732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef5024-3e38-4c43-81b0-b38efc4461b8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:31.777818 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 141/287] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:31 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:31.783488 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebea22-f13b-4c62-b300-0f835a0cf626 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:31.927384 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebea22-f13b-4c62-b300-0f835a0cf626 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:31.990968 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdae9c-f2c0-41ea-a1d8-8a8a1cfcb19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:31.994961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdae9c-f2c0-41ea-a1d8-8a8a1cfcb19b None admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:31.994961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdae9c-f2c0-41ea-a1d8-8a8a1cfcb19b None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:31.994961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdae9c-f2c0-41ea-a1d8-8a8a1cfcb19b None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551/users/39a48bea33ef4fc3a474aef8cdecdcc6/roles/74955b19ba90422cbe29fa461c3f287f` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:31.994961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdae9c-f2c0-41ea-a1d8-8a8a1cfcb19b None admin] RBAC: Authorizing `identity:create_grant(project_id=39c302b22dd04479bf8305109d613551, user_id=39a48bea33ef4fc3a474aef8cdecdcc6, role_id=74955b19ba90422cbe29fa461c3f287f)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:31.997818 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdae9c-f2c0-41ea-a1d8-8a8a1cfcb19b None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:32.082043 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 147/288] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:53:31 2022] PUT /identity/v3/projects/39c302b22dd04479bf8305109d613551/users/39a48bea33ef4fc3a474aef8cdecdcc6/roles/74955b19ba90422cbe29fa461c3f287f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:53:32.169249 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef5024-3e38-4c43-81b0-b38efc4461b8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:32.221154 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:32.306513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef5024-3e38-4c43-81b0-b38efc4461b8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:32.354252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4bdc9-8561-41f1-a2b4-98d77d630ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:32.354896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4bdc9-8561-41f1-a2b4-98d77d630ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:32.355195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4bdc9-8561-41f1-a2b4-98d77d630ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:32.355465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4bdc9-8561-41f1-a2b4-98d77d630ee2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:32.375715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4bdc9-8561-41f1-a2b4-98d77d630ee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:32.381401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-11d4bdc9-8561-41f1-a2b4-98d77d630ee2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:32.381401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:32.381401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:32.381401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:32.381401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:32.381401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:32.381401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:32.381401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:32.381401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:32.381401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:32.381401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:32.381401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:32.381401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:32.381401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:32.381401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:32.381401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:32.382298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:32.382298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:32.382298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:32.382298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:32.382298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:32.382298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:32.382298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:32.382298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:32.382298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:32.382298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:32.382298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:32.382298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:32.382298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:32.382298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:32.382298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:32.382298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:32.383119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:32.383119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:32.383119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:32.383119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:32.383119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:32.383119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:32.383119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:32.383119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:32.383119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:32.383119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:32.383119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:32.383119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:32.383119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 142/289] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:32.388938 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdae9c-f2c0-41ea-a1d8-8a8a1cfcb19b None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:32.420584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:32.496830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdae9c-f2c0-41ea-a1d8-8a8a1cfcb19b None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:32.533504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adfed42-75f1-4f88-81f7-46e22aef8699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:32.534108 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfed42-75f1-4f88-81f7-46e22aef8699 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:32.534382 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfed42-75f1-4f88-81f7-46e22aef8699 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:32.534650 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfed42-75f1-4f88-81f7-46e22aef8699 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:32.535347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adfed42-75f1-4f88-81f7-46e22aef8699 None admin] RBAC: Adding query filter params (name=member) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:32.535644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adfed42-75f1-4f88-81f7-46e22aef8699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:32.537017 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adfed42-75f1-4f88-81f7-46e22aef8699 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:32.546212 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 148/290] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:32.552040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4bdc9-8561-41f1-a2b4-98d77d630ee2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:32.682746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4bdc9-8561-41f1-a2b4-98d77d630ee2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:32.685163 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:32.735724 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67538486-1459-4e5c-b1d0-0104e6676058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:32.736358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67538486-1459-4e5c-b1d0-0104e6676058 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:32.736627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67538486-1459-4e5c-b1d0-0104e6676058 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:32.736892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67538486-1459-4e5c-b1d0-0104e6676058 None admin] PATH_INFO: `/v3/users/111855efac094907a56d3a29d5f61ee8` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:32.738025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67538486-1459-4e5c-b1d0-0104e6676058 None admin] RBAC: Authorizing `identity:get_user(user_id=111855efac094907a56d3a29d5f61ee8)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:32.739495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67538486-1459-4e5c-b1d0-0104e6676058 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:32.741053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 143/291] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:32 2022] GET /identity/v3/users/111855efac094907a56d3a29d5f61ee8 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:32.747412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adfed42-75f1-4f88-81f7-46e22aef8699 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:32.873741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:32.890467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adfed42-75f1-4f88-81f7-46e22aef8699 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:32.932855 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc11feb-8e69-4f2a-81e9-efed4a8fc924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:32.933464 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc11feb-8e69-4f2a-81e9-efed4a8fc924 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:32.933736 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc11feb-8e69-4f2a-81e9-efed4a8fc924 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:32.934000 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc11feb-8e69-4f2a-81e9-efed4a8fc924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:33.027445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc11feb-8e69-4f2a-81e9-efed4a8fc924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:33.028842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc11feb-8e69-4f2a-81e9-efed4a8fc924 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:33.125943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:33.140400 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 149/292] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:33.153654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67538486-1459-4e5c-b1d0-0104e6676058 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:33.263412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67538486-1459-4e5c-b1d0-0104e6676058 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:33.302834 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b479979-1f1e-4932-a9a9-0a08c73acc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:33.303893 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b479979-1f1e-4932-a9a9-0a08c73acc7c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:33.304401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b479979-1f1e-4932-a9a9-0a08c73acc7c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:33.304864 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b479979-1f1e-4932-a9a9-0a08c73acc7c None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:33.307771 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b479979-1f1e-4932-a9a9-0a08c73acc7c None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:33.309239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b479979-1f1e-4932-a9a9-0a08c73acc7c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:33.311016 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 144/293] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:33 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:33.317258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc11feb-8e69-4f2a-81e9-efed4a8fc924 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:33.433331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befbbaa-3309-42f2-929b-858c2f79a461 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:33.433652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befbbaa-3309-42f2-929b-858c2f79a461 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:33.433864 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befbbaa-3309-42f2-929b-858c2f79a461 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:33.435388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 145/294] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:33.440512 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4be00-398c-4801-8b48-fa859e892554 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:33.440783 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4be00-398c-4801-8b48-fa859e892554 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:33.441125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4be00-398c-4801-8b48-fa859e892554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:33.462752 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-03f4be00-398c-4801-8b48-fa859e892554 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:33.464407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-03f4be00-398c-4801-8b48-fa859e892554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:33.480641 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc11feb-8e69-4f2a-81e9-efed4a8fc924 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:33.542192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:33.547653 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af36396-7346-44a1-956b-8b9665902521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:33.548327 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af36396-7346-44a1-956b-8b9665902521 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:33.548604 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af36396-7346-44a1-956b-8b9665902521 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:33.548872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af36396-7346-44a1-956b-8b9665902521 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:33.549801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af36396-7346-44a1-956b-8b9665902521 None admin] RBAC: Adding query filter params (role.id=dd5cafa908a74cfd8e1a8baa44122242, scope.project.id=39c302b22dd04479bf8305109d613551, user.id=111855efac094907a56d3a29d5f61ee8) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:33.550104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af36396-7346-44a1-956b-8b9665902521 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:33.551603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af36396-7346-44a1-956b-8b9665902521 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:33.558877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 150/295] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:33 2022] GET /identity/v3/role_assignments?user.id=111855efac094907a56d3a29d5f61ee8&scope.project.id=39c302b22dd04479bf8305109d613551&role.id=dd5cafa908a74cfd8e1a8baa44122242 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:33.567004 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aecee10-15a2-43e8-ba34-de3ff592cc38 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:33.567343 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aecee10-15a2-43e8-ba34-de3ff592cc38 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:33.567539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aecee10-15a2-43e8-ba34-de3ff592cc38 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:33.574974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 151/296] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:33 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:33.575475 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4046817-5c0f-496a-845a-16bf635f9b65 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:33.575578 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4046817-5c0f-496a-845a-16bf635f9b65 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:33.575826 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4046817-5c0f-496a-845a-16bf635f9b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:33.594740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 146/297] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:33.604426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-c4046817-5c0f-496a-845a-16bf635f9b65 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:33.605798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-c4046817-5c0f-496a-845a-16bf635f9b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:33.707579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:33.761166 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 152/298] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:33.980654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4be00-398c-4801-8b48-fa859e892554 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:34.102339 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a52fd-0315-4739-9d08-325ea794a34a None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:34.102728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a52fd-0315-4739-9d08-325ea794a34a None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:34.103001 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a52fd-0315-4739-9d08-325ea794a34a None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:34.109820 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 153/299] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:53:34.111420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685d732-3eaf-4302-a20a-a2ceab7c2ec7 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:34.111703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685d732-3eaf-4302-a20a-a2ceab7c2ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:34.111973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685d732-3eaf-4302-a20a-a2ceab7c2ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:34.129040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4be00-398c-4801-8b48-fa859e892554 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:34.136398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-8685d732-3eaf-4302-a20a-a2ceab7c2ec7 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:34.137598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-8685d732-3eaf-4302-a20a-a2ceab7c2ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:34.192498 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46507a-0406-4905-a241-cff06611bac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:34.193140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46507a-0406-4905-a241-cff06611bac7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:34.193413 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46507a-0406-4905-a241-cff06611bac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:34.193845 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46507a-0406-4905-a241-cff06611bac7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:34.196359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46507a-0406-4905-a241-cff06611bac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:34.197812 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46507a-0406-4905-a241-cff06611bac7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:34.199755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 147/300] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:53:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:34.200595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46507a-0406-4905-a241-cff06611bac7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:34.233649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 154/301] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:34.234783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685d732-3eaf-4302-a20a-a2ceab7c2ec7 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:34.265513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:34.331530 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46507a-0406-4905-a241-cff06611bac7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:34.367534 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace3e35-6b78-4f61-9c20-da0d2910432a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:34.368230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace3e35-6b78-4f61-9c20-da0d2910432a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:34.368480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace3e35-6b78-4f61-9c20-da0d2910432a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:34.368706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace3e35-6b78-4f61-9c20-da0d2910432a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:34.371163 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685d732-3eaf-4302-a20a-a2ceab7c2ec7 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:34.380028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eace3e35-6b78-4f61-9c20-da0d2910432a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:34.385112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-eace3e35-6b78-4f61-9c20-da0d2910432a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:34.385112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:34.385112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:34.385112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:34.385112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:34.385112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:34.385112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:34.385112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:34.385112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:34.385112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:34.385112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:34.385112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:34.385112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:34.385112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:34.385112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:34.385112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:34.385989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:34.385989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:34.385989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:34.385989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:34.385989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:34.385989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:34.385989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:34.385989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:34.385989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:34.385989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:34.385989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:34.385989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:34.385989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:34.385989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:34.385989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:34.385989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:34.386822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:34.386822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:34.386822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:34.386822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:34.386822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:34.386822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:34.386822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:34.386822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:34.386822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:34.386822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:34.386822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:34.386822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:34.386822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 148/302] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:34.389754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:34.391963 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace3e35-6b78-4f61-9c20-da0d2910432a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:34.414111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250f374-9e42-44b1-9dbf-152484ca401a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:34.415141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250f374-9e42-44b1-9dbf-152484ca401a None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:34.415707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250f374-9e42-44b1-9dbf-152484ca401a None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:34.416170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250f374-9e42-44b1-9dbf-152484ca401a None admin] PATH_INFO: `/v3/groups` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:34.416980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250f374-9e42-44b1-9dbf-152484ca401a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:34.419062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250f374-9e42-44b1-9dbf-152484ca401a None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:34.492435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:34.496418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace3e35-6b78-4f61-9c20-da0d2910432a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:34.528053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 155/303] 10.222.0.30 () {60 vars in 1134 bytes} [Thu Oct 6 07:53:34 2022] POST /identity/v3/groups => generated 229 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:53:34.529311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250f374-9e42-44b1-9dbf-152484ca401a None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:34.532638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015eef0-afbb-46ec-8210-90f1c64c7e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:34.533310 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015eef0-afbb-46ec-8210-90f1c64c7e12 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:34.533586 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015eef0-afbb-46ec-8210-90f1c64c7e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:34.533856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015eef0-afbb-46ec-8210-90f1c64c7e12 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:34.534527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015eef0-afbb-46ec-8210-90f1c64c7e12 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:34.534827 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015eef0-afbb-46ec-8210-90f1c64c7e12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:34.536305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015eef0-afbb-46ec-8210-90f1c64c7e12 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:34.551233 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 149/304] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:34.552240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015eef0-afbb-46ec-8210-90f1c64c7e12 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:34.652567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250f374-9e42-44b1-9dbf-152484ca401a None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:34.678135 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015eef0-afbb-46ec-8210-90f1c64c7e12 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:34.694292 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21136a78-fe4f-4c6a-ae31-632c3f3be895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:34.695316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21136a78-fe4f-4c6a-ae31-632c3f3be895 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:34.695894 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21136a78-fe4f-4c6a-ae31-632c3f3be895 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:34.696376 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21136a78-fe4f-4c6a-ae31-632c3f3be895 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:34.709174 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21136a78-fe4f-4c6a-ae31-632c3f3be895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:34.716593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-21136a78-fe4f-4c6a-ae31-632c3f3be895 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:34.716593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:34.716593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:34.716593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:34.716593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:34.716593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:34.716593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:34.716593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:34.716593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:34.716593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:34.716593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:34.716593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:34.716593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:34.716593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:34.716593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:34.716593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:34.717809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:34.717809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:34.717809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:34.717809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:34.717809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:34.717809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:34.717809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:34.717809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:34.717809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:34.717809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:34.717809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:34.717809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:34.717809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:34.717809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:34.717809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:34.717809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:34.718615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:34.718615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:34.718615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:34.718615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:34.718615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:34.718615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:34.718615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:34.718615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:34.718615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:34.718615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:34.718615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:34.718615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:34.721850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 156/305] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:34.728256 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99f476-f9bf-4329-8e2f-122ef53df847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:34.728631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21136a78-fe4f-4c6a-ae31-632c3f3be895 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:34.728908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99f476-f9bf-4329-8e2f-122ef53df847 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:34.729188 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99f476-f9bf-4329-8e2f-122ef53df847 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:34.729459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99f476-f9bf-4329-8e2f-122ef53df847 None admin] PATH_INFO: `/v3/users/511d12a70200442f827d37fa6fa980af` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:34.744279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99f476-f9bf-4329-8e2f-122ef53df847 None admin] RBAC: Authorizing `identity:get_user(user_id=511d12a70200442f827d37fa6fa980af)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:34.745787 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99f476-f9bf-4329-8e2f-122ef53df847 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:34.748126 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 150/306] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:34 2022] GET /identity/v3/users/511d12a70200442f827d37fa6fa980af => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:34.753533 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99f476-f9bf-4329-8e2f-122ef53df847 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:34.770266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:34.775410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:34.854392 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21136a78-fe4f-4c6a-ae31-632c3f3be895 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:34.880979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99f476-f9bf-4329-8e2f-122ef53df847 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:34.891266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ed715-e8b1-45e0-87d8-ca12f9444d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:34.891723 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ed715-e8b1-45e0-87d8-ca12f9444d33 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:34.891997 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ed715-e8b1-45e0-87d8-ca12f9444d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:34.892267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ed715-e8b1-45e0-87d8-ca12f9444d33 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:34.892926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ed715-e8b1-45e0-87d8-ca12f9444d33 None admin] RBAC: Adding query filter params (name=member) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:34.893284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ed715-e8b1-45e0-87d8-ca12f9444d33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:34.894681 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ed715-e8b1-45e0-87d8-ca12f9444d33 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:34.908361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 157/307] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:34.910514 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ed715-e8b1-45e0-87d8-ca12f9444d33 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:34.930375 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7861620-1dcb-4e43-a393-2a022ab85e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:34.930985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7861620-1dcb-4e43-a393-2a022ab85e3c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:34.931264 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7861620-1dcb-4e43-a393-2a022ab85e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:34.931527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7861620-1dcb-4e43-a393-2a022ab85e3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:34.932254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7861620-1dcb-4e43-a393-2a022ab85e3c None admin] RBAC: Adding query filter params (role.id=74955b19ba90422cbe29fa461c3f287f, user.id=511d12a70200442f827d37fa6fa980af, scope.system=all) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:34.932553 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7861620-1dcb-4e43-a393-2a022ab85e3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:34.934153 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7861620-1dcb-4e43-a393-2a022ab85e3c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:34.940853 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 151/308] 10.222.0.30 () {56 vars in 1308 bytes} [Thu Oct 6 07:53:34 2022] GET /identity/v3/role_assignments?user.id=511d12a70200442f827d37fa6fa980af&scope.system=all&role.id=74955b19ba90422cbe29fa461c3f287f => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:34.948704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35e01a-9fb3-4dbf-ab6b-eaf71eee7425 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:34.949339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35e01a-9fb3-4dbf-ab6b-eaf71eee7425 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:34.949837 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35e01a-9fb3-4dbf-ab6b-eaf71eee7425 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:34.952894 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 152/309] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:34.958529 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9181fb3-d71a-4b23-8efe-146a5dae89fe None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:34.958827 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9181fb3-d71a-4b23-8efe-146a5dae89fe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:34.959113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9181fb3-d71a-4b23-8efe-146a5dae89fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:34.979059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b9181fb3-d71a-4b23-8efe-146a5dae89fe None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:34.980288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b9181fb3-d71a-4b23-8efe-146a5dae89fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:35.050038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:35.060084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 153/310] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:35.060701 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ed715-e8b1-45e0-87d8-ca12f9444d33 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:35.100195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578d9d3-2df9-42c8-9205-21d21f2fdbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:35.100853 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578d9d3-2df9-42c8-9205-21d21f2fdbac None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:35.101133 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578d9d3-2df9-42c8-9205-21d21f2fdbac None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:35.101407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578d9d3-2df9-42c8-9205-21d21f2fdbac None admin] PATH_INFO: `/v3/users/76106b84b2564b359b48c9c9fe5993a2` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:35.102673 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578d9d3-2df9-42c8-9205-21d21f2fdbac None admin] RBAC: Authorizing `identity:get_user(user_id=76106b84b2564b359b48c9c9fe5993a2)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:35.104221 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578d9d3-2df9-42c8-9205-21d21f2fdbac None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:35.106297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 158/311] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:34 2022] GET /identity/v3/users/76106b84b2564b359b48c9c9fe5993a2 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:35.111945 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9181fb3-d71a-4b23-8efe-146a5dae89fe None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:35.128908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:35.229759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9181fb3-d71a-4b23-8efe-146a5dae89fe None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:35.265648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a21acd-567d-4b63-8e16-b257fadda36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:35.266671 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a21acd-567d-4b63-8e16-b257fadda36c None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:35.267178 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a21acd-567d-4b63-8e16-b257fadda36c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:35.267714 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a21acd-567d-4b63-8e16-b257fadda36c None admin] PATH_INFO: `/v3/system/users/76106b84b2564b359b48c9c9fe5993a2/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:35.269339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a21acd-567d-4b63-8e16-b257fadda36c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=76106b84b2564b359b48c9c9fe5993a2, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:35.270995 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a21acd-567d-4b63-8e16-b257fadda36c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:35.337901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578d9d3-2df9-42c8-9205-21d21f2fdbac None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:35.390603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 154/312] 10.222.0.30 () {58 vars in 1334 bytes} [Thu Oct 6 07:53:35 2022] PUT /identity/v3/system/users/76106b84b2564b359b48c9c9fe5993a2/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:53:35.462164 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578d9d3-2df9-42c8-9205-21d21f2fdbac None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:35.503214 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea1085-28b0-434f-aace-37f60d945561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:35.503924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea1085-28b0-434f-aace-37f60d945561 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:35.504205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea1085-28b0-434f-aace-37f60d945561 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:35.504476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea1085-28b0-434f-aace-37f60d945561 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:35.516986 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea1085-28b0-434f-aace-37f60d945561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:35.522705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-9aea1085-28b0-434f-aace-37f60d945561 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:35.522705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:35.522705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:35.522705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:35.522705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:35.522705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:35.522705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:35.522705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:35.522705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:35.522705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:35.522705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:35.522705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:35.522705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:35.522705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:35.522705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:35.522705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:35.524402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:35.524402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:35.524402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:35.524402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:35.524402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:35.524402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:35.524402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:35.524402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:35.524402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:35.524402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:35.524402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:35.524402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:35.524402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:35.524402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:35.524402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:35.524402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:35.525795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:35.525795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:35.525795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:35.525795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:35.525795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:35.525795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:35.525795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:35.525795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:35.525795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:35.525795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:35.525795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:35.525795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:35.525795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 159/313] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:53:35.535915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a21acd-567d-4b63-8e16-b257fadda36c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:35.629564 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:35.658029 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a21acd-567d-4b63-8e16-b257fadda36c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:35.693606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f181a1-3427-440e-a4b4-610f7b0f5e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:35.694253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f181a1-3427-440e-a4b4-610f7b0f5e55 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:35.694463 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f181a1-3427-440e-a4b4-610f7b0f5e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:35.694727 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f181a1-3427-440e-a4b4-610f7b0f5e55 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:35.695420 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f181a1-3427-440e-a4b4-610f7b0f5e55 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:35.696334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f181a1-3427-440e-a4b4-610f7b0f5e55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:35.697840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f181a1-3427-440e-a4b4-610f7b0f5e55 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:35.706640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 155/314] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:35.712951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea1085-28b0-434f-aace-37f60d945561 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:35.738892 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:35.818571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea1085-28b0-434f-aace-37f60d945561 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:35.855186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a20ba-701d-4661-b8b6-ba900a584254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:35.855823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a20ba-701d-4661-b8b6-ba900a584254 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:35.856102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a20ba-701d-4661-b8b6-ba900a584254 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:35.856376 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a20ba-701d-4661-b8b6-ba900a584254 None admin] PATH_INFO: `/v3/users/39a48bea33ef4fc3a474aef8cdecdcc6` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:35.857526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a20ba-701d-4661-b8b6-ba900a584254 None admin] RBAC: Authorizing `identity:get_user(user_id=39a48bea33ef4fc3a474aef8cdecdcc6)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:35.859008 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a20ba-701d-4661-b8b6-ba900a584254 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:35.860980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 160/315] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:35 2022] GET /identity/v3/users/39a48bea33ef4fc3a474aef8cdecdcc6 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:35.866232 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f181a1-3427-440e-a4b4-610f7b0f5e55 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:35.973479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f181a1-3427-440e-a4b4-610f7b0f5e55 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:36.009717 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6942a6-aa09-4f83-a3ac-f0aa45e1665b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:36.011491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6942a6-aa09-4f83-a3ac-f0aa45e1665b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:36.011973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6942a6-aa09-4f83-a3ac-f0aa45e1665b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:36.012472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6942a6-aa09-4f83-a3ac-f0aa45e1665b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:36.035300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:36.083117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6942a6-aa09-4f83-a3ac-f0aa45e1665b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:36.084695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6942a6-aa09-4f83-a3ac-f0aa45e1665b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:36.189838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 156/316] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:36.195544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a20ba-701d-4661-b8b6-ba900a584254 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:36.303680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:36.313669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a20ba-701d-4661-b8b6-ba900a584254 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:36.350133 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca63d2-44ee-4774-a8f0-253079c4836a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:36.350849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca63d2-44ee-4774-a8f0-253079c4836a None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:36.351175 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca63d2-44ee-4774-a8f0-253079c4836a None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:36.351448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca63d2-44ee-4774-a8f0-253079c4836a None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:36.353437 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca63d2-44ee-4774-a8f0-253079c4836a None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:36.354982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca63d2-44ee-4774-a8f0-253079c4836a None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:36.357106 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 161/317] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:36 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:36.363166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6942a6-aa09-4f83-a3ac-f0aa45e1665b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:36.405392 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:36.467443 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6942a6-aa09-4f83-a3ac-f0aa45e1665b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:36.501603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648b24b-f448-4586-9f65-7e4bada43846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:36.502562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648b24b-f448-4586-9f65-7e4bada43846 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:36.503064 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648b24b-f448-4586-9f65-7e4bada43846 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:36.503527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648b24b-f448-4586-9f65-7e4bada43846 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:36.504430 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648b24b-f448-4586-9f65-7e4bada43846 None admin] RBAC: Adding query filter params (role.id=74955b19ba90422cbe29fa461c3f287f, scope.project.id=39c302b22dd04479bf8305109d613551, user.id=39a48bea33ef4fc3a474aef8cdecdcc6) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:36.504927 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648b24b-f448-4586-9f65-7e4bada43846 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:36.506583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648b24b-f448-4586-9f65-7e4bada43846 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:36.513509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 157/318] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:53:36 2022] GET /identity/v3/role_assignments?user.id=39a48bea33ef4fc3a474aef8cdecdcc6&scope.project.id=39c302b22dd04479bf8305109d613551&role.id=74955b19ba90422cbe29fa461c3f287f => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:36.944268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13cdf4-50fc-4af9-acad-14d31ff43e56 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:36.944780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13cdf4-50fc-4af9-acad-14d31ff43e56 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:36.944855 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13cdf4-50fc-4af9-acad-14d31ff43e56 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:36.946861 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 162/319] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:53:36.951968 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43f1ac-9d20-41d2-ba1d-f2dec37a124b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:36.952359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43f1ac-9d20-41d2-ba1d-f2dec37a124b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:36.952528 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43f1ac-9d20-41d2-ba1d-f2dec37a124b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:36.970871 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cd43f1ac-9d20-41d2-ba1d-f2dec37a124b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:36.972082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cd43f1ac-9d20-41d2-ba1d-f2dec37a124b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:37.040168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 158/320] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:37.218477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f1f4e-b4ee-4288-bffe-0d3a1db1febc None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:37.218783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f1f4e-b4ee-4288-bffe-0d3a1db1febc None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:37.218993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f1f4e-b4ee-4288-bffe-0d3a1db1febc None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:37.220467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 163/321] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:37.225515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc506bb-873d-460c-b44b-d7daa03282ae None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:37.225801 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc506bb-873d-460c-b44b-d7daa03282ae None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:37.226072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc506bb-873d-460c-b44b-d7daa03282ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:37.246319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cbc506bb-873d-460c-b44b-d7daa03282ae None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:37.247526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cbc506bb-873d-460c-b44b-d7daa03282ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:37.273494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:37.309992 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f1f4e-b4ee-4288-bffe-0d3a1db1febc None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:37.321270 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 159/322] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:37.415506 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f1f4e-b4ee-4288-bffe-0d3a1db1febc None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:37.449262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cfdd2-ebc4-43df-afd8-dfaece385764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:37.449888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cfdd2-ebc4-43df-afd8-dfaece385764 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:37.450166 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cfdd2-ebc4-43df-afd8-dfaece385764 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:37.450435 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cfdd2-ebc4-43df-afd8-dfaece385764 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:37.451553 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9cfdd2-ebc4-43df-afd8-dfaece385764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:37.453020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9cfdd2-ebc4-43df-afd8-dfaece385764 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:37.454971 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 164/323] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:53:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:37.460080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc506bb-873d-460c-b44b-d7daa03282ae None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:37.565624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc506bb-873d-460c-b44b-d7daa03282ae None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:37.589329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cfdd2-ebc4-43df-afd8-dfaece385764 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:37.602426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f5925-4ec7-4a2f-b0d0-586bb561477c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:37.603338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f5925-4ec7-4a2f-b0d0-586bb561477c None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:37.603612 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f5925-4ec7-4a2f-b0d0-586bb561477c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:37.603879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f5925-4ec7-4a2f-b0d0-586bb561477c None admin] PATH_INFO: `/v3/groups` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:37.604690 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f5925-4ec7-4a2f-b0d0-586bb561477c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:37.606545 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f5925-4ec7-4a2f-b0d0-586bb561477c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:37.616995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:37.688680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cfdd2-ebc4-43df-afd8-dfaece385764 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:37.709865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 160/324] 10.222.0.30 () {60 vars in 1134 bytes} [Thu Oct 6 07:53:37 2022] POST /identity/v3/groups => generated 226 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:53:37.713433 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09267ee-80b5-46c8-8817-585b20d97bd8 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:37.713716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09267ee-80b5-46c8-8817-585b20d97bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:37.713987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09267ee-80b5-46c8-8817-585b20d97bd8 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:37.715525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 161/325] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:37.721511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf91c3-8ad8-4c22-be11-a1e0c53b7fdf None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:37.721911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf91c3-8ad8-4c22-be11-a1e0c53b7fdf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:37.722302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf91c3-8ad8-4c22-be11-a1e0c53b7fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:37.723489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b89ce-820e-48fc-b0e3-ce70fa2e81f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:37.724147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b89ce-820e-48fc-b0e3-ce70fa2e81f5 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:37.724421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b89ce-820e-48fc-b0e3-ce70fa2e81f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:37.724689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b89ce-820e-48fc-b0e3-ce70fa2e81f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:37.738540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b89ce-820e-48fc-b0e3-ce70fa2e81f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:37.743716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-019b89ce-820e-48fc-b0e3-ce70fa2e81f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:37.743716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:37.743716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:37.743716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:37.743716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:37.743716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:37.743716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:37.743716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:37.743716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:37.743716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:37.743716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:37.743716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:37.743716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:37.743716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:37.743716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:37.743716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:37.744696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:37.744696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:37.744696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:37.744696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:37.744696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:37.744696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:37.744696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:37.744696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:37.744696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:37.744696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:37.744696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:37.744696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:37.744696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:37.744696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:37.744696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:37.744696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:37.747922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:37.747922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:37.747922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:37.747922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:37.747922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:37.747922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:37.747922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:37.747922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:37.747922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:37.747922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:37.747922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:37.747922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:37.749156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f1cf91c3-8ad8-4c22-be11-a1e0c53b7fdf None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:37.749156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f1cf91c3-8ad8-4c22-be11-a1e0c53b7fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:37.749294 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 165/326] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:53:37.751344 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b89ce-820e-48fc-b0e3-ce70fa2e81f5 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:37.764998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:37.835099 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 162/327] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:37.852632 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:37.868553 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b89ce-820e-48fc-b0e3-ce70fa2e81f5 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:37.904935 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747c279-3cb8-4c75-ac34-a76d1a1cc879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:37.905672 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747c279-3cb8-4c75-ac34-a76d1a1cc879 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:37.905951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747c279-3cb8-4c75-ac34-a76d1a1cc879 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:37.906223 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747c279-3cb8-4c75-ac34-a76d1a1cc879 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:37.906886 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747c279-3cb8-4c75-ac34-a76d1a1cc879 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:37.907227 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747c279-3cb8-4c75-ac34-a76d1a1cc879 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:37.908602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747c279-3cb8-4c75-ac34-a76d1a1cc879 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:37.918515 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 166/328] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:37.924893 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf91c3-8ad8-4c22-be11-a1e0c53b7fdf None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:38.033418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf91c3-8ad8-4c22-be11-a1e0c53b7fdf None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:38.074056 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe873f43-f6d5-48f1-b154-fc8da0aa4111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:38.074709 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe873f43-f6d5-48f1-b154-fc8da0aa4111 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:38.075623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe873f43-f6d5-48f1-b154-fc8da0aa4111 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:38.075623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe873f43-f6d5-48f1-b154-fc8da0aa4111 None admin] PATH_INFO: `/v3/users/511d12a70200442f827d37fa6fa980af` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:38.076768 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe873f43-f6d5-48f1-b154-fc8da0aa4111 None admin] RBAC: Authorizing `identity:get_user(user_id=511d12a70200442f827d37fa6fa980af)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:38.078237 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe873f43-f6d5-48f1-b154-fc8da0aa4111 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:38.081288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 163/329] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:37 2022] GET /identity/v3/users/511d12a70200442f827d37fa6fa980af => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:38.086999 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747c279-3cb8-4c75-ac34-a76d1a1cc879 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:38.127577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe873f43-f6d5-48f1-b154-fc8da0aa4111 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:38.172779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:38.231829 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747c279-3cb8-4c75-ac34-a76d1a1cc879 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:38.272716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb2647-27e5-4950-a1ed-4ffd2ad0a94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:38.273374 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb2647-27e5-4950-a1ed-4ffd2ad0a94e None admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:38.273707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb2647-27e5-4950-a1ed-4ffd2ad0a94e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:38.274570 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb2647-27e5-4950-a1ed-4ffd2ad0a94e None admin] PATH_INFO: `/v3/system/users/511d12a70200442f827d37fa6fa980af/roles/74955b19ba90422cbe29fa461c3f287f` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:38.276153 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb2647-27e5-4950-a1ed-4ffd2ad0a94e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=511d12a70200442f827d37fa6fa980af, role_id=74955b19ba90422cbe29fa461c3f287f)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:38.277524 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb2647-27e5-4950-a1ed-4ffd2ad0a94e None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:38.279407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe873f43-f6d5-48f1-b154-fc8da0aa4111 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:38.321275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97b87f-f1c0-4695-b74d-170949e1bfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:38.321962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97b87f-f1c0-4695-b74d-170949e1bfa6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:38.322248 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97b87f-f1c0-4695-b74d-170949e1bfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:38.322587 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97b87f-f1c0-4695-b74d-170949e1bfa6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:38.333944 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97b87f-f1c0-4695-b74d-170949e1bfa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:38.339352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-cb97b87f-f1c0-4695-b74d-170949e1bfa6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:38.339352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:38.339352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:38.339352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:38.339352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:38.339352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:38.339352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:38.339352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:38.339352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:38.339352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:38.339352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:38.339352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:38.339352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:38.339352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:38.339352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:38.339352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:38.340266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:38.340266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:38.340266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:38.340266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:38.340266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:38.340266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:38.340266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:38.340266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:38.340266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:38.340266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:38.340266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:38.340266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:38.340266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:38.340266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:38.340266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:38.340266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:38.341239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:38.341239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:38.341239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:38.341239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:38.341239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:38.341239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:38.341239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:38.341239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:38.341239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:38.341239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:38.341239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:38.341239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:38.341239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 164/330] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:53:38.344736 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 167/331] 10.222.0.30 () {58 vars in 1334 bytes} [Thu Oct 6 07:53:38 2022] PUT /identity/v3/system/users/511d12a70200442f827d37fa6fa980af/roles/74955b19ba90422cbe29fa461c3f287f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:53:38.346972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97b87f-f1c0-4695-b74d-170949e1bfa6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:38.449027 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:38.472986 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97b87f-f1c0-4695-b74d-170949e1bfa6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:38.521017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2692686-8fe9-4087-8d4d-5072784cb3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:38.521640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2692686-8fe9-4087-8d4d-5072784cb3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:38.521917 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2692686-8fe9-4087-8d4d-5072784cb3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:38.522190 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2692686-8fe9-4087-8d4d-5072784cb3f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:38.522869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2692686-8fe9-4087-8d4d-5072784cb3f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:38.523246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2692686-8fe9-4087-8d4d-5072784cb3f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:38.525469 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2692686-8fe9-4087-8d4d-5072784cb3f5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:38.536945 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 165/332] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:38.543364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb2647-27e5-4950-a1ed-4ffd2ad0a94e None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:38.563137 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:38.665627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb2647-27e5-4950-a1ed-4ffd2ad0a94e None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:38.705523 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa5e30-54bf-4ca9-bc16-5c338cf07344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:38.706157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa5e30-54bf-4ca9-bc16-5c338cf07344 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:38.706429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa5e30-54bf-4ca9-bc16-5c338cf07344 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:38.706694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa5e30-54bf-4ca9-bc16-5c338cf07344 None admin] PATH_INFO: `/v3/users/76106b84b2564b359b48c9c9fe5993a2` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:38.707869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa5e30-54bf-4ca9-bc16-5c338cf07344 None admin] RBAC: Authorizing `identity:get_user(user_id=76106b84b2564b359b48c9c9fe5993a2)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:38.709319 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa5e30-54bf-4ca9-bc16-5c338cf07344 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:38.711349 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 168/333] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:38 2022] GET /identity/v3/users/76106b84b2564b359b48c9c9fe5993a2 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:38.716940 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2692686-8fe9-4087-8d4d-5072784cb3f5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:38.853696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2692686-8fe9-4087-8d4d-5072784cb3f5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:38.900548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff62c2-dc5d-4b6a-aac3-35b7e129ccf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:38.901162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff62c2-dc5d-4b6a-aac3-35b7e129ccf0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:38.901439 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff62c2-dc5d-4b6a-aac3-35b7e129ccf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:38.901705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff62c2-dc5d-4b6a-aac3-35b7e129ccf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:38.902439 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff62c2-dc5d-4b6a-aac3-35b7e129ccf0 None admin] RBAC: Adding query filter params (role.id=dd5cafa908a74cfd8e1a8baa44122242, user.id=76106b84b2564b359b48c9c9fe5993a2, scope.system=all) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:38.902743 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff62c2-dc5d-4b6a-aac3-35b7e129ccf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:38.904199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff62c2-dc5d-4b6a-aac3-35b7e129ccf0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:38.920543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:38.926972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 166/334] 10.222.0.30 () {56 vars in 1308 bytes} [Thu Oct 6 07:53:38 2022] GET /identity/v3/role_assignments?user.id=76106b84b2564b359b48c9c9fe5993a2&scope.system=all&role.id=dd5cafa908a74cfd8e1a8baa44122242 => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:40.491098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c668457-92ad-436d-872a-59a125d430fe None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:40.492148 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c668457-92ad-436d-872a-59a125d430fe None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:40.492773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c668457-92ad-436d-872a-59a125d430fe None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:40.494452 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 169/335] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:40.500968 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a9054-96a7-47aa-b24a-192c2013a7bf None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:40.502015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a9054-96a7-47aa-b24a-192c2013a7bf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:40.502538 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a9054-96a7-47aa-b24a-192c2013a7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:40.525890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fe1a9054-96a7-47aa-b24a-192c2013a7bf None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:40.527440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fe1a9054-96a7-47aa-b24a-192c2013a7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:40.604278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 167/336] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:40.741577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c948e9c-c0c7-4e58-99dd-bc46f4279290 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:40.742918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c948e9c-c0c7-4e58-99dd-bc46f4279290 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:40.743410 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c948e9c-c0c7-4e58-99dd-bc46f4279290 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:40.744972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 170/337] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:40.750630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac69e86-edc7-449b-8d83-b1a3aa9612e3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:40.751172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac69e86-edc7-449b-8d83-b1a3aa9612e3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:40.751673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac69e86-edc7-449b-8d83-b1a3aa9612e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:40.775937 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-aac69e86-edc7-449b-8d83-b1a3aa9612e3 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:40.777460 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-aac69e86-edc7-449b-8d83-b1a3aa9612e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:40.860797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 168/338] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:41.060917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c948e9c-c0c7-4e58-99dd-bc46f4279290 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:41.305647 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb03714-fa4f-48e9-9631-0022fd7e9554 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:41.306456 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb03714-fa4f-48e9-9631-0022fd7e9554 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:41.306992 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb03714-fa4f-48e9-9631-0022fd7e9554 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:41.308579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 169/339] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:53:41.313984 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb0b28-63bc-474c-9508-f91ff87f7ba6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:41.314495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb0b28-63bc-474c-9508-f91ff87f7ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:41.315084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb0b28-63bc-474c-9508-f91ff87f7ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:41.347742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dabb0b28-63bc-474c-9508-f91ff87f7ba6 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:41.349985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dabb0b28-63bc-474c-9508-f91ff87f7ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:41.395289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:41.425561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c948e9c-c0c7-4e58-99dd-bc46f4279290 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:41.456421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:41.471486 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070aee81-1048-43c4-8e05-290d354825b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:41.472433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070aee81-1048-43c4-8e05-290d354825b2 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:41.472901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070aee81-1048-43c4-8e05-290d354825b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:41.473410 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070aee81-1048-43c4-8e05-290d354825b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:41.481564 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 170/340] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:41.483255 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2f94a-a769-4b0a-abb9-d0b4022e3f64 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:41.483539 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2f94a-a769-4b0a-abb9-d0b4022e3f64 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:41.483811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2f94a-a769-4b0a-abb9-d0b4022e3f64 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:41.485062 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 171/341] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:41.487059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2f94a-a769-4b0a-abb9-d0b4022e3f64 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:41.489813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070aee81-1048-43c4-8e05-290d354825b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:41.495616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-070aee81-1048-43c4-8e05-290d354825b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:41.495616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:41.495616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:41.495616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:41.495616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:41.495616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:41.495616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:41.495616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:41.495616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:41.495616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:41.495616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:41.495616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:41.495616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:41.495616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:41.495616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:41.495616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:41.496972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:41.496972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:41.496972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:41.496972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:41.496972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:41.496972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:41.496972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:41.496972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:41.496972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:41.496972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:41.496972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:41.496972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:41.496972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:41.496972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:41.496972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:41.496972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:41.497754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:41.497754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:41.497754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:41.497754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:41.497754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:41.497754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:41.497754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:41.497754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:41.497754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:41.497754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:41.497754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:41.497754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:41.501434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 171/342] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:53:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 441 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:41.502836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ae8bf-db6c-496b-9d42-26bb38bbe717 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:41.503352 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ae8bf-db6c-496b-9d42-26bb38bbe717 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:41.503818 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ae8bf-db6c-496b-9d42-26bb38bbe717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:41.522754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-572ae8bf-db6c-496b-9d42-26bb38bbe717 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:41.525297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-572ae8bf-db6c-496b-9d42-26bb38bbe717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:41.628543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 172/343] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:41.629196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ae8bf-db6c-496b-9d42-26bb38bbe717 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:41.681269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2f94a-a769-4b0a-abb9-d0b4022e3f64 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:41.725210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388dae1-f74c-41d7-940c-f141008c7687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:41.726212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388dae1-f74c-41d7-940c-f141008c7687 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:41.726745 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388dae1-f74c-41d7-940c-f141008c7687 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:41.727828 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388dae1-f74c-41d7-940c-f141008c7687 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:41.740191 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388dae1-f74c-41d7-940c-f141008c7687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:41.746393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-2388dae1-f74c-41d7-940c-f141008c7687 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:41.746393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:41.746393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:41.746393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:41.746393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:41.746393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:41.746393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:41.746393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:41.746393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:41.746393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:41.746393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:41.746393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:41.746393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:41.746393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:41.746393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:41.746393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:41.747516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:41.747516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:41.747516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:41.747516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:41.747516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:41.747516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:41.747516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:41.747516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:41.747516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:41.747516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:41.747516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:41.747516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:41.747516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:41.747516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:41.747516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:41.747516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:41.748308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:41.748308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:41.748308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:41.748308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:41.748308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:41.748308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:41.748308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:41.748308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:41.748308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:41.748308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:41.748308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:41.748308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:41.751496 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 172/344] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:41.759771 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388dae1-f74c-41d7-940c-f141008c7687 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:41.763427 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:41.834677 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ae8bf-db6c-496b-9d42-26bb38bbe717 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:41.839955 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:41.907240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a972a04-8dc0-40fd-b292-066db097c6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:41.908200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a972a04-8dc0-40fd-b292-066db097c6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:41.908666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a972a04-8dc0-40fd-b292-066db097c6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:41.909136 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a972a04-8dc0-40fd-b292-066db097c6b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:41.910052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a972a04-8dc0-40fd-b292-066db097c6b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:41.910565 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a972a04-8dc0-40fd-b292-066db097c6b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:41.912980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a972a04-8dc0-40fd-b292-066db097c6b4 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:41.933299 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 173/345] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:53:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:41.934452 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a972a04-8dc0-40fd-b292-066db097c6b4 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:41.941380 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388dae1-f74c-41d7-940c-f141008c7687 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:41.997390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f9a89-9def-4eb7-a60f-e58b6836ad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:41.998669 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f9a89-9def-4eb7-a60f-e58b6836ad45 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:42.000065 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f9a89-9def-4eb7-a60f-e58b6836ad45 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:42.001237 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f9a89-9def-4eb7-a60f-e58b6836ad45 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:42.002102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f9a89-9def-4eb7-a60f-e58b6836ad45 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:42.004048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f9a89-9def-4eb7-a60f-e58b6836ad45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:42.005619 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f9a89-9def-4eb7-a60f-e58b6836ad45 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:42.025272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 173/346] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:42.028586 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f9a89-9def-4eb7-a60f-e58b6836ad45 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:42.107051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a972a04-8dc0-40fd-b292-066db097c6b4 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:42.149511 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:42.171741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407b28f-135a-47ba-ba45-0a872b3a53ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:42.171741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407b28f-135a-47ba-ba45-0a872b3a53ec None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:42.171741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407b28f-135a-47ba-ba45-0a872b3a53ec None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:42.171741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407b28f-135a-47ba-ba45-0a872b3a53ec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:42.177333 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f9a89-9def-4eb7-a60f-e58b6836ad45 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:42.180109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407b28f-135a-47ba-ba45-0a872b3a53ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:42.185749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-9407b28f-135a-47ba-ba45-0a872b3a53ec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:42.185749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:42.185749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:42.185749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:42.185749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:42.185749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:42.185749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:42.185749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:42.185749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:42.185749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:42.185749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:42.185749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:42.185749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:42.185749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:42.185749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:42.185749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:42.186670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:42.186670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:42.186670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:42.186670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:42.186670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:42.186670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:42.186670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:42.186670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:42.186670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:42.186670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:42.186670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:42.186670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:42.186670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:42.186670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:42.186670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:42.186670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:42.187618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:42.187618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:42.187618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:42.187618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:42.187618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:42.187618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:42.187618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:42.187618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:42.187618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:42.187618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:42.187618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 06 07:53:42.187618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:42.194972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 174/347] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:42.196259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:42.197596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407b28f-135a-47ba-ba45-0a872b3a53ec None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:42.222263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdeded8-3f0b-46ce-a6fd-fd2ed33c7180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:42.222870 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdeded8-3f0b-46ce-a6fd-fd2ed33c7180 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:42.223174 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdeded8-3f0b-46ce-a6fd-fd2ed33c7180 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:42.223442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdeded8-3f0b-46ce-a6fd-fd2ed33c7180 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:42.319102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdeded8-3f0b-46ce-a6fd-fd2ed33c7180 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:42.322253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdeded8-3f0b-46ce-a6fd-fd2ed33c7180 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:42.327784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407b28f-135a-47ba-ba45-0a872b3a53ec None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:42.365179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf26b55-b15a-4ad5-98f2-362b74b5b30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:42.365788 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf26b55-b15a-4ad5-98f2-362b74b5b30c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:42.366069 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf26b55-b15a-4ad5-98f2-362b74b5b30c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:42.366342 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf26b55-b15a-4ad5-98f2-362b74b5b30c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:42.433776 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:42.441674 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 174/348] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:42.446416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf26b55-b15a-4ad5-98f2-362b74b5b30c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:42.447767 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf26b55-b15a-4ad5-98f2-362b74b5b30c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:42.452604 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdeded8-3f0b-46ce-a6fd-fd2ed33c7180 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:42.465745 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:42.563947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdeded8-3f0b-46ce-a6fd-fd2ed33c7180 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:42.564718 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 175/349] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:42.565673 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf26b55-b15a-4ad5-98f2-362b74b5b30c None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:42.621202 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d91d0f-cf62-4856-8fba-8bdd5f21ff27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:42.621838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d91d0f-cf62-4856-8fba-8bdd5f21ff27 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:42.622110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d91d0f-cf62-4856-8fba-8bdd5f21ff27 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:42.622381 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d91d0f-cf62-4856-8fba-8bdd5f21ff27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:42.634268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d91d0f-cf62-4856-8fba-8bdd5f21ff27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:42.639111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c7d91d0f-cf62-4856-8fba-8bdd5f21ff27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:42.639111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:42.639111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:42.639111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:42.639111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:42.639111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:42.639111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:42.639111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:42.639111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:42.639111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:42.639111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:42.639111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:42.639111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:42.639111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:42.639111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:42.639111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:42.640066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:42.640066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:42.640066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:42.640066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:42.640066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:42.640066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:42.640066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:42.640066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:42.640066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:42.640066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:42.640066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:42.640066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:42.640066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:42.640066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:42.640066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:42.640066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:42.640911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:42.640911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:42.640911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:42.640911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:42.640911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:42.640911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:42.640911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:42.640911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:42.640911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:42.640911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:42.640911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:42.640911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:42.640911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 175/350] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:42.641631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d91d0f-cf62-4856-8fba-8bdd5f21ff27 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:42.691225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:42.706418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf26b55-b15a-4ad5-98f2-362b74b5b30c None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:42.735068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:42.753418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9da21-39dc-45a6-94c9-283caef0d76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:42.753418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9da21-39dc-45a6-94c9-283caef0d76b None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:42.753418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9da21-39dc-45a6-94c9-283caef0d76b None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:42.753418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9da21-39dc-45a6-94c9-283caef0d76b None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:42.753887 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9da21-39dc-45a6-94c9-283caef0d76b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:42.753887 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9da21-39dc-45a6-94c9-283caef0d76b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:42.754735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9da21-39dc-45a6-94c9-283caef0d76b None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:42.766777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 176/351] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:42.772731 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9365d-3d48-4495-aa87-52fdb9aa6f17 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:42.773015 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9365d-3d48-4495-aa87-52fdb9aa6f17 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:42.773340 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9365d-3d48-4495-aa87-52fdb9aa6f17 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:42.774876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 177/352] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:42.775432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9365d-3d48-4495-aa87-52fdb9aa6f17 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:42.788909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d91d0f-cf62-4856-8fba-8bdd5f21ff27 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:42.840471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad768c1d-61dd-46c3-80a8-23b251034662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:42.840471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad768c1d-61dd-46c3-80a8-23b251034662 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:42.840471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad768c1d-61dd-46c3-80a8-23b251034662 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:42.840471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad768c1d-61dd-46c3-80a8-23b251034662 None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:42.842149 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad768c1d-61dd-46c3-80a8-23b251034662 None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:42.843898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad768c1d-61dd-46c3-80a8-23b251034662 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:42.845547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 176/353] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:42 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:42.846308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad768c1d-61dd-46c3-80a8-23b251034662 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:42.900371 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9365d-3d48-4495-aa87-52fdb9aa6f17 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:42.940454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efb957-4e48-49e8-97e2-f31fb7f38e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:42.941182 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efb957-4e48-49e8-97e2-f31fb7f38e91 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:42.941504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efb957-4e48-49e8-97e2-f31fb7f38e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:42.941724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efb957-4e48-49e8-97e2-f31fb7f38e91 None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:42.942969 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10efb957-4e48-49e8-97e2-f31fb7f38e91 None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:42.944461 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10efb957-4e48-49e8-97e2-f31fb7f38e91 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:42.946776 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 178/354] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:42 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:42.949390 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efb957-4e48-49e8-97e2-f31fb7f38e91 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:42.980810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad768c1d-61dd-46c3-80a8-23b251034662 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:42.983446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:43.024821 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbb70e-7ad0-42f3-867f-376ba80dc39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:43.025771 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbb70e-7ad0-42f3-867f-376ba80dc39e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:43.026240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbb70e-7ad0-42f3-867f-376ba80dc39e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:43.026709 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbb70e-7ad0-42f3-867f-376ba80dc39e None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:43.027788 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbb70e-7ad0-42f3-867f-376ba80dc39e None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:43.028680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbb70e-7ad0-42f3-867f-376ba80dc39e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:43.030458 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbb70e-7ad0-42f3-867f-376ba80dc39e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:43.042352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 177/355] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:43.043650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68882848-0731-4017-8d76-feeeb62b339c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:43.043929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68882848-0731-4017-8d76-feeeb62b339c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:43.044197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68882848-0731-4017-8d76-feeeb62b339c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:43.052500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:43.068060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-68882848-0731-4017-8d76-feeeb62b339c None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:43.069295 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-68882848-0731-4017-8d76-feeeb62b339c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:43.094389 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efb957-4e48-49e8-97e2-f31fb7f38e91 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:43.154830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a87819-8b91-4fa1-a21b-0cb2e16765f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:43.155501 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a87819-8b91-4fa1-a21b-0cb2e16765f3 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:43.156274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a87819-8b91-4fa1-a21b-0cb2e16765f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:43.156274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a87819-8b91-4fa1-a21b-0cb2e16765f3 None admin] PATH_INFO: `/v3/users/511d12a70200442f827d37fa6fa980af` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:43.157634 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a87819-8b91-4fa1-a21b-0cb2e16765f3 None admin] RBAC: Authorizing `identity:get_user(user_id=511d12a70200442f827d37fa6fa980af)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:43.160201 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a87819-8b91-4fa1-a21b-0cb2e16765f3 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:43.162584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 179/356] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:53:42 2022] GET /identity/v3/users/511d12a70200442f827d37fa6fa980af => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:43.163824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a87819-8b91-4fa1-a21b-0cb2e16765f3 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:43.172072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 178/357] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:43.172919 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68882848-0731-4017-8d76-feeeb62b339c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:43.280186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:43.299229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a87819-8b91-4fa1-a21b-0cb2e16765f3 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:43.306814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68882848-0731-4017-8d76-feeeb62b339c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:43.347331 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825b6b4-a1ff-4ee9-9c11-e226b238d6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:43.347794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825b6b4-a1ff-4ee9-9c11-e226b238d6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:43.348070 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825b6b4-a1ff-4ee9-9c11-e226b238d6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:43.348341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825b6b4-a1ff-4ee9-9c11-e226b238d6a4 None admin] PATH_INFO: `/v3/groups/9952ba9059344824b23210e7f3d962ab` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:43.355941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:43.357638 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825b6b4-a1ff-4ee9-9c11-e226b238d6a4 None admin] RBAC: Authorizing `identity:get_group(group_id=9952ba9059344824b23210e7f3d962ab)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:43.359763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825b6b4-a1ff-4ee9-9c11-e226b238d6a4 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:43.361617 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 180/358] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Oct 6 07:53:43 2022] GET /identity/v3/groups/9952ba9059344824b23210e7f3d962ab => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:43.366428 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825b6b4-a1ff-4ee9-9c11-e226b238d6a4 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:43.368706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf763918-c841-4186-9759-4225d35eec78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:43.369344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf763918-c841-4186-9759-4225d35eec78 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:43.369622 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf763918-c841-4186-9759-4225d35eec78 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:43.369890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf763918-c841-4186-9759-4225d35eec78 None admin] PATH_INFO: `/v3/groups/9952ba9059344824b23210e7f3d962ab` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:43.371874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf763918-c841-4186-9759-4225d35eec78 None admin] RBAC: Authorizing `identity:get_group(group_id=9952ba9059344824b23210e7f3d962ab)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:43.373820 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf763918-c841-4186-9759-4225d35eec78 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:43.375771 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 179/359] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Oct 6 07:53:43 2022] GET /identity/v3/groups/9952ba9059344824b23210e7f3d962ab => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:43.376762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf763918-c841-4186-9759-4225d35eec78 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:43.491803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825b6b4-a1ff-4ee9-9c11-e226b238d6a4 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:43.516000 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf763918-c841-4186-9759-4225d35eec78 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:43.537635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decefd5-0125-423d-b82e-13703b170313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:43.538783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decefd5-0125-423d-b82e-13703b170313 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:43.538783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decefd5-0125-423d-b82e-13703b170313 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:43.538783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decefd5-0125-423d-b82e-13703b170313 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:43.553811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5e640-5864-49b4-a4d6-8911c0dbf99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:43.554417 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5e640-5864-49b4-a4d6-8911c0dbf99c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:43.555972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5e640-5864-49b4-a4d6-8911c0dbf99c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:43.556251 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5e640-5864-49b4-a4d6-8911c0dbf99c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:43.556985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5e640-5864-49b4-a4d6-8911c0dbf99c None admin] RBAC: Adding query filter params (role.id=74955b19ba90422cbe29fa461c3f287f, user.id=511d12a70200442f827d37fa6fa980af, scope.system=all) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:43.557294 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5e640-5864-49b4-a4d6-8911c0dbf99c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:43.558746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5e640-5864-49b4-a4d6-8911c0dbf99c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:43.567724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:43.573667 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 180/360] 10.222.0.30 () {56 vars in 1308 bytes} [Thu Oct 6 07:53:43 2022] GET /identity/v3/role_assignments?user.id=511d12a70200442f827d37fa6fa980af&scope.system=all&role.id=74955b19ba90422cbe29fa461c3f287f => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:43.575221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb78ff-af14-468b-9104-368a11f732fe None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:43.575507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb78ff-af14-468b-9104-368a11f732fe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:43.576122 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb78ff-af14-468b-9104-368a11f732fe None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:43.577579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 181/361] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:43.578226 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb78ff-af14-468b-9104-368a11f732fe None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:43.624930 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2decefd5-0125-423d-b82e-13703b170313 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:43.626567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2decefd5-0125-423d-b82e-13703b170313 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:43.675178 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:43.747920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb78ff-af14-468b-9104-368a11f732fe None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:43.777171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 181/362] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:43.778118 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decefd5-0125-423d-b82e-13703b170313 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:43.797517 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2af7f-8fb6-436b-ad6a-5d0276eb93dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:43.798051 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2af7f-8fb6-436b-ad6a-5d0276eb93dc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:43.798325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2af7f-8fb6-436b-ad6a-5d0276eb93dc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:43.798592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2af7f-8fb6-436b-ad6a-5d0276eb93dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:43.799929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2af7f-8fb6-436b-ad6a-5d0276eb93dc None admin] RBAC: Adding query filter params (group.id=9952ba9059344824b23210e7f3d962ab, role.id=d9b2237fd9884cc39771e75bf902e203, scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:43.800367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2af7f-8fb6-436b-ad6a-5d0276eb93dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:43.802509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2af7f-8fb6-436b-ad6a-5d0276eb93dc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:43.809994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 182/363] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Oct 6 07:53:43 2022] GET /identity/v3/role_assignments?group.id=9952ba9059344824b23210e7f3d962ab&scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45&role.id=d9b2237fd9884cc39771e75bf902e203 => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:43.810177 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2af7f-8fb6-436b-ad6a-5d0276eb93dc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:43.857164 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:43.916144 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decefd5-0125-423d-b82e-13703b170313 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:43.962281 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ff27a-5b2a-4fb7-9479-7a75a3c6cdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:43.962913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2af7f-8fb6-436b-ad6a-5d0276eb93dc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:43.963494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ff27a-5b2a-4fb7-9479-7a75a3c6cdfb None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:43.963774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ff27a-5b2a-4fb7-9479-7a75a3c6cdfb None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:43.964041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ff27a-5b2a-4fb7-9479-7a75a3c6cdfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:43.964814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ff27a-5b2a-4fb7-9479-7a75a3c6cdfb None admin] RBAC: Adding query filter params (group.id=9952ba9059344824b23210e7f3d962ab, role.id=dd5cafa908a74cfd8e1a8baa44122242, scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:43.965117 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ff27a-5b2a-4fb7-9479-7a75a3c6cdfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:43.967021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ff27a-5b2a-4fb7-9479-7a75a3c6cdfb None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:43.974737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 182/364] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Oct 6 07:53:43 2022] GET /identity/v3/role_assignments?group.id=9952ba9059344824b23210e7f3d962ab&scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45&role.id=dd5cafa908a74cfd8e1a8baa44122242 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:43.980271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665e68a-1a8c-4540-83b4-f83cfa699eb5 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:43.980271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665e68a-1a8c-4540-83b4-f83cfa699eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:43.980271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665e68a-1a8c-4540-83b4-f83cfa699eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:44.000710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-0665e68a-1a8c-4540-83b4-f83cfa699eb5 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:44.002230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0665e68a-1a8c-4540-83b4-f83cfa699eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:44.004107 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:44.032409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41f152-a38c-476a-b2d7-22ff2b509687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:44.033573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41f152-a38c-476a-b2d7-22ff2b509687 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:44.033573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41f152-a38c-476a-b2d7-22ff2b509687 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:44.033573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41f152-a38c-476a-b2d7-22ff2b509687 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:44.046495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41f152-a38c-476a-b2d7-22ff2b509687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:44.054410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-8d41f152-a38c-476a-b2d7-22ff2b509687 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:44.054410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:44.054410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:44.054410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:44.054410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:44.054410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:44.054410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:44.054410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:44.054410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:44.054410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:44.054410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:44.054410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:44.054410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:44.054410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:44.054410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:44.054410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:44.056077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:44.056077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:44.056077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:44.056077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:44.056077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:44.056077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:44.056077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:44.056077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:44.056077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:44.056077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:44.056077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:44.056077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:44.056077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:44.056077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:44.056077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:44.056077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:44.057374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:44.057374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:44.057374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:44.057374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:44.057374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:44.057374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:44.057374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:44.057374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:44.057374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:44.057374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:44.057374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:44.057374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:44.057374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 183/365] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:53:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:44.057374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41f152-a38c-476a-b2d7-22ff2b509687 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:44.112423 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 183/366] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:44.113313 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665e68a-1a8c-4540-83b4-f83cfa699eb5 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:44.197021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:44.260422 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41f152-a38c-476a-b2d7-22ff2b509687 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:44.268697 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665e68a-1a8c-4540-83b4-f83cfa699eb5 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:44.309736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2cf36-9994-4734-bc6d-56347a413696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:44.310292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2cf36-9994-4734-bc6d-56347a413696 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:44.310564 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2cf36-9994-4734-bc6d-56347a413696 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:44.310831 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2cf36-9994-4734-bc6d-56347a413696 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:44.311990 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2cf36-9994-4734-bc6d-56347a413696 None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:44.313449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2cf36-9994-4734-bc6d-56347a413696 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:44.319214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 184/367] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:44 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:44.320554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2cf36-9994-4734-bc6d-56347a413696 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:44.328496 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf3ba6-cbff-4831-8b86-c79a2de4705e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:44.329126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf3ba6-cbff-4831-8b86-c79a2de4705e None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:44.329400 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf3ba6-cbff-4831-8b86-c79a2de4705e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:44.329667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf3ba6-cbff-4831-8b86-c79a2de4705e None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:44.330344 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf3ba6-cbff-4831-8b86-c79a2de4705e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:44.330646 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf3ba6-cbff-4831-8b86-c79a2de4705e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:44.332093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf3ba6-cbff-4831-8b86-c79a2de4705e None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:44.348834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 184/368] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:53:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:44.356351 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf3ba6-cbff-4831-8b86-c79a2de4705e None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:44.390411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:44.460035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2cf36-9994-4734-bc6d-56347a413696 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:44.507138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6ff6a-eb94-4b68-b390-6a1d9be95135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:44.509274 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6ff6a-eb94-4b68-b390-6a1d9be95135 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:44.509950 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6ff6a-eb94-4b68-b390-6a1d9be95135 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:44.510385 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6ff6a-eb94-4b68-b390-6a1d9be95135 None admin] PATH_INFO: `/v3/groups/9952ba9059344824b23210e7f3d962ab` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:44.516673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6ff6a-eb94-4b68-b390-6a1d9be95135 None admin] RBAC: Authorizing `identity:get_group(group_id=9952ba9059344824b23210e7f3d962ab)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:44.518143 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6ff6a-eb94-4b68-b390-6a1d9be95135 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:44.519980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 185/369] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Oct 6 07:53:44 2022] GET /identity/v3/groups/9952ba9059344824b23210e7f3d962ab => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:44.520560 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6ff6a-eb94-4b68-b390-6a1d9be95135 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:44.523434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf3ba6-cbff-4831-8b86-c79a2de4705e None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:44.538121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:44.563881 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a06b7-69fd-4f96-9ccc-c4f1a1488aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:44.564675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a06b7-69fd-4f96-9ccc-c4f1a1488aba None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:44.564785 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a06b7-69fd-4f96-9ccc-c4f1a1488aba None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:44.565040 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a06b7-69fd-4f96-9ccc-c4f1a1488aba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:44.651847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6ff6a-eb94-4b68-b390-6a1d9be95135 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:44.656238 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a06b7-69fd-4f96-9ccc-c4f1a1488aba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:44.657726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a06b7-69fd-4f96-9ccc-c4f1a1488aba None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:44.674786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:44.698679 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f73ebc-05e5-46e6-bfd9-367c783ac020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:44.699754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f73ebc-05e5-46e6-bfd9-367c783ac020 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:44.700257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f73ebc-05e5-46e6-bfd9-367c783ac020 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:44.700730 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f73ebc-05e5-46e6-bfd9-367c783ac020 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:44.717168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f73ebc-05e5-46e6-bfd9-367c783ac020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:44.723720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-b9f73ebc-05e5-46e6-bfd9-367c783ac020 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:44.723720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:44.723720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:44.723720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:44.723720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:44.723720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:44.723720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:44.723720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:44.723720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:44.723720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:44.723720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:44.723720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:44.723720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:44.723720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:44.723720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:44.723720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:44.724654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:44.724654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:44.724654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:44.724654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:44.724654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:44.724654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:44.724654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:44.724654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:44.724654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:44.724654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:44.724654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:44.724654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:44.724654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:44.724654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:44.724654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:44.724654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:44.725459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:44.725459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:44.725459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:44.725459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:44.725459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:44.725459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:44.725459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:44.725459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:44.725459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:44.725459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:44.725459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:44.725459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:44.729083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 186/370] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:53:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:44.734079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f73ebc-05e5-46e6-bfd9-367c783ac020 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:44.785337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 185/371] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:44.786626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a06b7-69fd-4f96-9ccc-c4f1a1488aba None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:44.799741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:44.866455 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f73ebc-05e5-46e6-bfd9-367c783ac020 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:44.911001 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e748226-728e-4b02-9088-23729896c7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:44.911628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e748226-728e-4b02-9088-23729896c7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:44.911897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e748226-728e-4b02-9088-23729896c7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:44.912160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e748226-728e-4b02-9088-23729896c7f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:44.912896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e748226-728e-4b02-9088-23729896c7f9 None admin] RBAC: Adding query filter params (group.id=9952ba9059344824b23210e7f3d962ab, role.id=dd5cafa908a74cfd8e1a8baa44122242, scope.project.id=39c302b22dd04479bf8305109d613551) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:44.913201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e748226-728e-4b02-9088-23729896c7f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:44.915231 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e748226-728e-4b02-9088-23729896c7f9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:44.917258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a06b7-69fd-4f96-9ccc-c4f1a1488aba None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:44.924921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 187/372] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Oct 6 07:53:44 2022] GET /identity/v3/role_assignments?group.id=9952ba9059344824b23210e7f3d962ab&scope.project.id=39c302b22dd04479bf8305109d613551&role.id=dd5cafa908a74cfd8e1a8baa44122242 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:44.926159 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e748226-728e-4b02-9088-23729896c7f9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:44.976229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a980504-ca7a-46f2-b4de-ba6b969068af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:44.978917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a980504-ca7a-46f2-b4de-ba6b969068af None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:44.979457 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a980504-ca7a-46f2-b4de-ba6b969068af None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:44.981152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a980504-ca7a-46f2-b4de-ba6b969068af None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:44.981942 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a980504-ca7a-46f2-b4de-ba6b969068af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:44.982248 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a980504-ca7a-46f2-b4de-ba6b969068af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:44.983731 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a980504-ca7a-46f2-b4de-ba6b969068af None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:45.002566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 186/373] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:45.006529 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:45.009884 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a980504-ca7a-46f2-b4de-ba6b969068af None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:45.133924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e748226-728e-4b02-9088-23729896c7f9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:45.152844 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:45.187131 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71302e-e6b3-40dd-a0b5-32940db13ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:45.187757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71302e-e6b3-40dd-a0b5-32940db13ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:45.188080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71302e-e6b3-40dd-a0b5-32940db13ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:45.188345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71302e-e6b3-40dd-a0b5-32940db13ca2 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:45.189703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71302e-e6b3-40dd-a0b5-32940db13ca2 None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:45.190900 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71302e-e6b3-40dd-a0b5-32940db13ca2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:45.193259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 188/374] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:44 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:45.198047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a980504-ca7a-46f2-b4de-ba6b969068af None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:45.199921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71302e-e6b3-40dd-a0b5-32940db13ca2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:45.273158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969542d0-bff8-45fc-b04f-4989da2693ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:45.274297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969542d0-bff8-45fc-b04f-4989da2693ff None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:45.274719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969542d0-bff8-45fc-b04f-4989da2693ff None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:45.274992 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969542d0-bff8-45fc-b04f-4989da2693ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:45.379987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969542d0-bff8-45fc-b04f-4989da2693ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:45.381837 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969542d0-bff8-45fc-b04f-4989da2693ff None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:45.401564 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71302e-e6b3-40dd-a0b5-32940db13ca2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:45.427318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:45.468476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:45.494831 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fd5c6-12b2-4bf6-b41f-5872993b4104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:45.495507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fd5c6-12b2-4bf6-b41f-5872993b4104 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:45.495779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fd5c6-12b2-4bf6-b41f-5872993b4104 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:45.496049 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fd5c6-12b2-4bf6-b41f-5872993b4104 None admin] PATH_INFO: `/v3/groups/9952ba9059344824b23210e7f3d962ab` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:45.497282 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fd5c6-12b2-4bf6-b41f-5872993b4104 None admin] RBAC: Authorizing `identity:get_group(group_id=9952ba9059344824b23210e7f3d962ab)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:45.501272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fd5c6-12b2-4bf6-b41f-5872993b4104 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:45.505939 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 189/375] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Oct 6 07:53:45 2022] GET /identity/v3/groups/9952ba9059344824b23210e7f3d962ab => generated 226 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:45.511167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fd5c6-12b2-4bf6-b41f-5872993b4104 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:45.525659 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 187/376] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 07:53:45.531198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969542d0-bff8-45fc-b04f-4989da2693ff None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:45.705984 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fd5c6-12b2-4bf6-b41f-5872993b4104 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:45.719052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969542d0-bff8-45fc-b04f-4989da2693ff None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:45.772617 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca3fb3-1479-4d7a-b5b6-e2feaff3637f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:45.773354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca3fb3-1479-4d7a-b5b6-e2feaff3637f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:45.774152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca3fb3-1479-4d7a-b5b6-e2feaff3637f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:45.774411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca3fb3-1479-4d7a-b5b6-e2feaff3637f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:45.775136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca3fb3-1479-4d7a-b5b6-e2feaff3637f None admin] RBAC: Adding query filter params (group.id=9952ba9059344824b23210e7f3d962ab, role.id=d9b2237fd9884cc39771e75bf902e203, scope.project.id=39c302b22dd04479bf8305109d613551) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:45.775437 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca3fb3-1479-4d7a-b5b6-e2feaff3637f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:45.776857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca3fb3-1479-4d7a-b5b6-e2feaff3637f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:45.785387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 190/377] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Oct 6 07:53:45 2022] GET /identity/v3/role_assignments?group.id=9952ba9059344824b23210e7f3d962ab&scope.project.id=39c302b22dd04479bf8305109d613551&role.id=d9b2237fd9884cc39771e75bf902e203 => generated 260 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:45.789613 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6dd032-860a-4f35-9ddd-9cb243276342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:45.790248 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dd032-860a-4f35-9ddd-9cb243276342 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:45.790582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dd032-860a-4f35-9ddd-9cb243276342 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:45.790855 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dd032-860a-4f35-9ddd-9cb243276342 None admin] PATH_INFO: `/v3/projects/f2d30f50f03441f1a79068b287cb95c1` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:45.792058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6dd032-860a-4f35-9ddd-9cb243276342 None admin] RBAC: Authorizing `identity:get_project(project_id=f2d30f50f03441f1a79068b287cb95c1)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:45.793606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6dd032-860a-4f35-9ddd-9cb243276342 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:45.795797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 188/378] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:45 2022] GET /identity/v3/projects/f2d30f50f03441f1a79068b287cb95c1 => generated 344 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:45.803698 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca3fb3-1479-4d7a-b5b6-e2feaff3637f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:45.870998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:45.944153 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca3fb3-1479-4d7a-b5b6-e2feaff3637f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:45.985295 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3a894-93ad-4717-96a7-2ee9d16edae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:45.985884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3a894-93ad-4717-96a7-2ee9d16edae1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:45.986162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3a894-93ad-4717-96a7-2ee9d16edae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:45.986434 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3a894-93ad-4717-96a7-2ee9d16edae1 None admin] PATH_INFO: `/v3/groups/caa581df8bd54f6a86f3e0f8016b290c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:45.998142 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3a894-93ad-4717-96a7-2ee9d16edae1 None admin] RBAC: Authorizing `identity:get_group(group_id=caa581df8bd54f6a86f3e0f8016b290c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:45.999921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3a894-93ad-4717-96a7-2ee9d16edae1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:46.001947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 191/379] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Oct 6 07:53:45 2022] GET /identity/v3/groups/caa581df8bd54f6a86f3e0f8016b290c => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:46.008245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6dd032-860a-4f35-9ddd-9cb243276342 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:46.096456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:46.179911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6dd032-860a-4f35-9ddd-9cb243276342 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:46.225623 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb1d70-85aa-4845-80b0-5d661d35b5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:46.226222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb1d70-85aa-4845-80b0-5d661d35b5aa None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:46.227349 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb1d70-85aa-4845-80b0-5d661d35b5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:46.227678 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb1d70-85aa-4845-80b0-5d661d35b5aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:46.228511 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb1d70-85aa-4845-80b0-5d661d35b5aa None admin] RBAC: Adding query filter params (group.id=caa581df8bd54f6a86f3e0f8016b290c, role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=f2d30f50f03441f1a79068b287cb95c1) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:46.228817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb1d70-85aa-4845-80b0-5d661d35b5aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:46.230405 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb1d70-85aa-4845-80b0-5d661d35b5aa None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:46.239424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 189/380] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Oct 6 07:53:46 2022] GET /identity/v3/role_assignments?group.id=caa581df8bd54f6a86f3e0f8016b290c&scope.project.id=f2d30f50f03441f1a79068b287cb95c1&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:46.621864 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea7193-e3d2-480c-9079-df587e0738c1 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:46.622200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea7193-e3d2-480c-9079-df587e0738c1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:46.622313 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea7193-e3d2-480c-9079-df587e0738c1 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:46.624312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 192/381] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:46.629338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad0d1c-2f9d-4cf3-850b-35b88911b106 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:46.629622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad0d1c-2f9d-4cf3-850b-35b88911b106 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:46.629898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad0d1c-2f9d-4cf3-850b-35b88911b106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:46.656053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-4aad0d1c-2f9d-4cf3-850b-35b88911b106 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:46.657286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4aad0d1c-2f9d-4cf3-850b-35b88911b106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:46.698736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457d980-31ea-4542-b4e4-d23f752d7a39 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:46.699038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457d980-31ea-4542-b4e4-d23f752d7a39 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:46.699546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457d980-31ea-4542-b4e4-d23f752d7a39 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:46.701428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 193/382] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:46.706588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360e62f-22e1-46ef-8359-64afa9aa641e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:46.707160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360e62f-22e1-46ef-8359-64afa9aa641e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:46.707650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360e62f-22e1-46ef-8359-64afa9aa641e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:46.732137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a360e62f-22e1-46ef-8359-64afa9aa641e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:46.733345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a360e62f-22e1-46ef-8359-64afa9aa641e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:46.749099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 190/383] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:46.837226 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 194/384] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:47.028911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad0d1c-2f9d-4cf3-850b-35b88911b106 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:47.072346 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:47.160497 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad0d1c-2f9d-4cf3-850b-35b88911b106 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:47.202766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebeb20-fa61-4ceb-b695-a0223bf841c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:47.203455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebeb20-fa61-4ceb-b695-a0223bf841c4 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:47.203731 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebeb20-fa61-4ceb-b695-a0223bf841c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:47.203997 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebeb20-fa61-4ceb-b695-a0223bf841c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:47.223525 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eebeb20-fa61-4ceb-b695-a0223bf841c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:47.229179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-6eebeb20-fa61-4ceb-b695-a0223bf841c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:47.229179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:47.229179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:47.229179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:47.229179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:47.229179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:47.229179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:47.229179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:47.229179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:47.229179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:47.229179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:47.229179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:47.229179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:47.229179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:47.229179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:47.229179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:47.230102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:47.230102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:47.230102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:47.230102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:47.230102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:47.230102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:47.230102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:47.230102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:47.230102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:47.230102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:47.230102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:47.230102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:47.230102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:47.230102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:47.230102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:47.230102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:47.230903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:47.230903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:47.230903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:47.230903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:47.230903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:47.230903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:47.230903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:47.230903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:47.230903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:47.230903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:47.230903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:47.230903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:47.230903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 191/385] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:47.240052 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360e62f-22e1-46ef-8359-64afa9aa641e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:47.271283 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebeb20-fa61-4ceb-b695-a0223bf841c4 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:47.311395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:47.425172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360e62f-22e1-46ef-8359-64afa9aa641e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:47.426429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:47.434250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebeb20-fa61-4ceb-b695-a0223bf841c4 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:47.471365 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba364d-4997-4840-afee-08f3ceec4b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:47.472155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba364d-4997-4840-afee-08f3ceec4b1c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:47.472708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba364d-4997-4840-afee-08f3ceec4b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:47.473245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba364d-4997-4840-afee-08f3ceec4b1c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:47.486185 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba364d-4997-4840-afee-08f3ceec4b1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:47.493446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-e7ba364d-4997-4840-afee-08f3ceec4b1c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:47.493446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:47.493446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:47.493446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:47.493446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:47.493446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:47.493446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:47.493446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:47.493446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:47.493446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:47.493446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:47.493446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:47.493446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:47.493446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:47.493446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:47.493446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:47.494631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:47.494631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:47.494631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:47.494631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:47.494631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:47.494631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:47.494631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:47.494631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:47.494631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:47.494631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:47.494631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:47.494631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:47.494631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:47.494631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:47.494631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:47.494631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:47.495557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:47.495557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:47.495557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:47.495557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:47.495557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:47.495557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:47.495557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:47.495557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:47.495557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:47.495557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:47.495557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:47.495557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:47.496188 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 192/386] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:53:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:53:47.502870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba364d-4997-4840-afee-08f3ceec4b1c None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:47.520464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dca376-14bf-4676-8df2-5abb69d6c084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:47.521399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dca376-14bf-4676-8df2-5abb69d6c084 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:47.521884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dca376-14bf-4676-8df2-5abb69d6c084 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:47.522363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dca376-14bf-4676-8df2-5abb69d6c084 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:47.523423 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dca376-14bf-4676-8df2-5abb69d6c084 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:47.524120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dca376-14bf-4676-8df2-5abb69d6c084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:47.525782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dca376-14bf-4676-8df2-5abb69d6c084 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:47.545486 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 195/387] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:47.552296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dca376-14bf-4676-8df2-5abb69d6c084 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:47.675177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba364d-4997-4840-afee-08f3ceec4b1c None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:47.700963 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:47.705806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dca376-14bf-4676-8df2-5abb69d6c084 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:47.718052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bef9c-0ee3-4730-ae91-b61431332a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:47.718967 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bef9c-0ee3-4730-ae91-b61431332a4c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:47.719264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bef9c-0ee3-4730-ae91-b61431332a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:47.719535 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bef9c-0ee3-4730-ae91-b61431332a4c None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:47.720199 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bef9c-0ee3-4730-ae91-b61431332a4c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:47.720632 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bef9c-0ee3-4730-ae91-b61431332a4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:47.722164 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bef9c-0ee3-4730-ae91-b61431332a4c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:47.738116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 193/388] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:53:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:47.738116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5abde-75a1-4aeb-a1fe-3a0a20ce566a None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:47.738116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5abde-75a1-4aeb-a1fe-3a0a20ce566a None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:47.738116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5abde-75a1-4aeb-a1fe-3a0a20ce566a None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:47.738116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 194/389] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:47.738116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5abde-75a1-4aeb-a1fe-3a0a20ce566a None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:47.757609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781a20e-80a5-4aac-9349-a492c23cd789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:47.758592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781a20e-80a5-4aac-9349-a492c23cd789 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:47.759176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781a20e-80a5-4aac-9349-a492c23cd789 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:47.759638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781a20e-80a5-4aac-9349-a492c23cd789 None admin] PATH_INFO: `/v3/groups/9952ba9059344824b23210e7f3d962ab` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:47.761179 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781a20e-80a5-4aac-9349-a492c23cd789 None admin] RBAC: Authorizing `identity:get_group(group_id=9952ba9059344824b23210e7f3d962ab)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:47.762882 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781a20e-80a5-4aac-9349-a492c23cd789 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:47.764969 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 196/390] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Oct 6 07:53:47 2022] GET /identity/v3/groups/9952ba9059344824b23210e7f3d962ab => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:47.766306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badeded6-e1f8-4520-afbc-760e5fe0168c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:47.766779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badeded6-e1f8-4520-afbc-760e5fe0168c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:47.767297 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badeded6-e1f8-4520-afbc-760e5fe0168c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:47.779794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:47.788030 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-badeded6-e1f8-4520-afbc-760e5fe0168c None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:47.791198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-badeded6-e1f8-4520-afbc-760e5fe0168c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:47.876403 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5abde-75a1-4aeb-a1fe-3a0a20ce566a None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:47.878985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 197/391] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:47.878985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badeded6-e1f8-4520-afbc-760e5fe0168c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:47.943948 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864270f-febd-48e8-8713-d21ac98e57c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:47.949638 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864270f-febd-48e8-8713-d21ac98e57c0 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:47.949911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864270f-febd-48e8-8713-d21ac98e57c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:47.950179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864270f-febd-48e8-8713-d21ac98e57c0 None admin] PATH_INFO: `/v3/groups/9952ba9059344824b23210e7f3d962ab` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:47.953478 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864270f-febd-48e8-8713-d21ac98e57c0 None admin] RBAC: Authorizing `identity:get_group(group_id=9952ba9059344824b23210e7f3d962ab)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:47.954961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864270f-febd-48e8-8713-d21ac98e57c0 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:47.956540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 195/392] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Oct 6 07:53:47 2022] GET /identity/v3/groups/9952ba9059344824b23210e7f3d962ab => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:47.963947 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864270f-febd-48e8-8713-d21ac98e57c0 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:48.021782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:48.032729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badeded6-e1f8-4520-afbc-760e5fe0168c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:48.100650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13c439-d39a-4040-9e3e-e721f01e2712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:48.102124 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13c439-d39a-4040-9e3e-e721f01e2712 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:48.102397 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13c439-d39a-4040-9e3e-e721f01e2712 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:48.102721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13c439-d39a-4040-9e3e-e721f01e2712 None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:48.105144 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13c439-d39a-4040-9e3e-e721f01e2712 None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:48.106564 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13c439-d39a-4040-9e3e-e721f01e2712 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:48.108210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 198/393] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:47 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:48.115241 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13c439-d39a-4040-9e3e-e721f01e2712 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:48.126447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:48.156798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864270f-febd-48e8-8713-d21ac98e57c0 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:48.195894 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd4126-fc07-49f3-8ee7-79afa41872fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:48.196437 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd4126-fc07-49f3-8ee7-79afa41872fd None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:48.196710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd4126-fc07-49f3-8ee7-79afa41872fd None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:48.196977 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd4126-fc07-49f3-8ee7-79afa41872fd None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:48.198107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd4126-fc07-49f3-8ee7-79afa41872fd None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:48.199639 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd4126-fc07-49f3-8ee7-79afa41872fd None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:48.201889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 196/394] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:47 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:48.212209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd4126-fc07-49f3-8ee7-79afa41872fd None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:48.268773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13c439-d39a-4040-9e3e-e721f01e2712 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:48.308994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2928c78-a034-4319-bfc6-cffd1fd6914d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:48.309756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2928c78-a034-4319-bfc6-cffd1fd6914d None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:48.310040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2928c78-a034-4319-bfc6-cffd1fd6914d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:48.310308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2928c78-a034-4319-bfc6-cffd1fd6914d None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45/groups/9952ba9059344824b23210e7f3d962ab/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:48.312323 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2928c78-a034-4319-bfc6-cffd1fd6914d None admin] RBAC: Authorizing `identity:create_grant(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45, group_id=9952ba9059344824b23210e7f3d962ab, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:48.313888 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2928c78-a034-4319-bfc6-cffd1fd6914d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:48.337076 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd4126-fc07-49f3-8ee7-79afa41872fd None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:48.372492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d596a3-1b0f-4f16-b28c-348008cdd35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:48.373152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d596a3-1b0f-4f16-b28c-348008cdd35b None admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:48.373685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d596a3-1b0f-4f16-b28c-348008cdd35b None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:48.373961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d596a3-1b0f-4f16-b28c-348008cdd35b None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45/groups/9952ba9059344824b23210e7f3d962ab/roles/d9b2237fd9884cc39771e75bf902e203` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:48.376143 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d596a3-1b0f-4f16-b28c-348008cdd35b None admin] RBAC: Authorizing `identity:create_grant(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45, group_id=9952ba9059344824b23210e7f3d962ab, role_id=d9b2237fd9884cc39771e75bf902e203)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:48.377588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d596a3-1b0f-4f16-b28c-348008cdd35b None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:48.403112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 199/395] 10.222.0.30 () {58 vars in 1442 bytes} [Thu Oct 6 07:53:48 2022] PUT /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45/groups/9952ba9059344824b23210e7f3d962ab/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:53:48.403112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2928c78-a034-4319-bfc6-cffd1fd6914d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:48.434983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:48.460494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 197/396] 10.222.0.30 () {58 vars in 1442 bytes} [Thu Oct 6 07:53:48 2022] PUT /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45/groups/9952ba9059344824b23210e7f3d962ab/roles/d9b2237fd9884cc39771e75bf902e203 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:53:48.549284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2928c78-a034-4319-bfc6-cffd1fd6914d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:48.598698 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f02a1b-f801-4e56-a301-79bb1072e01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:48.599654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f02a1b-f801-4e56-a301-79bb1072e01c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:48.600105 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f02a1b-f801-4e56-a301-79bb1072e01c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:48.600521 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f02a1b-f801-4e56-a301-79bb1072e01c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:48.620123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f02a1b-f801-4e56-a301-79bb1072e01c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:48.625736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-62f02a1b-f801-4e56-a301-79bb1072e01c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:48.625736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:48.625736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:48.625736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:48.625736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:48.625736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:48.625736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:48.625736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:48.625736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:48.625736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:48.625736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:48.625736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:48.625736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:48.625736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:48.625736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:48.625736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:48.626910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:48.626910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:48.626910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:48.626910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:48.626910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:48.626910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:48.626910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:48.626910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:48.626910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:48.626910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:48.626910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:48.626910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:48.626910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:48.626910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:48.626910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:48.626910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:48.627979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:48.627979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:48.627979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:48.627979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:48.627979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:48.627979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:48.627979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:48.627979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:48.627979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:48.627979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:48.627979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:48.627979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:48.630224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 200/397] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:48.637062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d596a3-1b0f-4f16-b28c-348008cdd35b None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:48.655972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:48.695929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299474a-f845-43de-ba81-78dd0257e433 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:48.696263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299474a-f845-43de-ba81-78dd0257e433 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:48.696476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299474a-f845-43de-ba81-78dd0257e433 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:48.698124 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 201/398] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:53:48.703723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be828f10-7bff-4f6e-b23b-42d49b9896b0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:48.704046 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be828f10-7bff-4f6e-b23b-42d49b9896b0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:48.704265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be828f10-7bff-4f6e-b23b-42d49b9896b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:48.728756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-be828f10-7bff-4f6e-b23b-42d49b9896b0 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:48.729342 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-be828f10-7bff-4f6e-b23b-42d49b9896b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:48.766103 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d596a3-1b0f-4f16-b28c-348008cdd35b None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:48.814506 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5b75f-aa3c-4713-9373-a94d6ccb5971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:48.815955 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5b75f-aa3c-4713-9373-a94d6ccb5971 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:48.816465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5b75f-aa3c-4713-9373-a94d6ccb5971 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:48.816941 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5b75f-aa3c-4713-9373-a94d6ccb5971 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:48.817858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5b75f-aa3c-4713-9373-a94d6ccb5971 None admin] RBAC: Adding query filter params (name=member) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:48.818388 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5b75f-aa3c-4713-9373-a94d6ccb5971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:48.820209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5b75f-aa3c-4713-9373-a94d6ccb5971 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:48.830281 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 198/399] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:48.839741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5b75f-aa3c-4713-9373-a94d6ccb5971 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:48.861096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 202/400] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:48.934326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399de14b-625a-4cb3-8513-a412a5b0466e None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:48.934882 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399de14b-625a-4cb3-8513-a412a5b0466e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:48.935349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399de14b-625a-4cb3-8513-a412a5b0466e None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:48.936906 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 203/401] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:53:48.945136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2468d-d6b3-4abb-bed3-449d8bcb8a8d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:48.945449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2468d-d6b3-4abb-bed3-449d8bcb8a8d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:48.945680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2468d-d6b3-4abb-bed3-449d8bcb8a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:48.949109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:48.974384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-61e2468d-d6b3-4abb-bed3-449d8bcb8a8d None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:48.979136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-61e2468d-d6b3-4abb-bed3-449d8bcb8a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:49.000509 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5b75f-aa3c-4713-9373-a94d6ccb5971 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:49.002338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:49.052728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992cc90-2cdb-46a5-b0af-3c165178b5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:49.053756 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992cc90-2cdb-46a5-b0af-3c165178b5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:49.054281 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992cc90-2cdb-46a5-b0af-3c165178b5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:49.054759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992cc90-2cdb-46a5-b0af-3c165178b5f1 None admin] PATH_INFO: `/v3/groups/9952ba9059344824b23210e7f3d962ab` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:49.056141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3992cc90-2cdb-46a5-b0af-3c165178b5f1 None admin] RBAC: Authorizing `identity:get_group(group_id=9952ba9059344824b23210e7f3d962ab)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:49.057896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3992cc90-2cdb-46a5-b0af-3c165178b5f1 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:49.064118 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 199/402] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Oct 6 07:53:48 2022] GET /identity/v3/groups/9952ba9059344824b23210e7f3d962ab => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:49.065239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992cc90-2cdb-46a5-b0af-3c165178b5f1 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:49.090098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 204/403] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 06 07:53:49.175896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e2468d-d6b3-4abb-bed3-449d8bcb8a8d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:49.194258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992cc90-2cdb-46a5-b0af-3c165178b5f1 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:49.230458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c789412-3965-449d-b6b9-116ae59e4291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:49.231036 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c789412-3965-449d-b6b9-116ae59e4291 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:49.231320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c789412-3965-449d-b6b9-116ae59e4291 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:49.231598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c789412-3965-449d-b6b9-116ae59e4291 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:49.232734 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c789412-3965-449d-b6b9-116ae59e4291 None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:49.234185 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c789412-3965-449d-b6b9-116ae59e4291 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:49.236220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 200/404] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:49 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:49.247856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c789412-3965-449d-b6b9-116ae59e4291 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:49.267693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:49.298116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e2468d-d6b3-4abb-bed3-449d8bcb8a8d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:49.359128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75085c0-a755-4daf-9e48-b3cc5d8ac6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:49.359771 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75085c0-a755-4daf-9e48-b3cc5d8ac6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:49.360047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75085c0-a755-4daf-9e48-b3cc5d8ac6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:49.360385 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75085c0-a755-4daf-9e48-b3cc5d8ac6f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:49.383098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c789412-3965-449d-b6b9-116ae59e4291 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:49.385447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75085c0-a755-4daf-9e48-b3cc5d8ac6f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:49.390387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d75085c0-a755-4daf-9e48-b3cc5d8ac6f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:49.390387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:49.390387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:49.390387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:49.390387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:49.390387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:49.390387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:49.390387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:49.390387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:49.390387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:49.390387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:49.390387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:49.390387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:49.390387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:49.390387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:49.390387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:49.391312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:49.391312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:49.391312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:49.391312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:49.391312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:49.391312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:49.391312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:49.391312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:49.391312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:49.391312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:49.391312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:49.391312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:49.391312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:49.391312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:49.391312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:49.391312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:49.392137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:49.392137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:49.392137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:49.392137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:49.392137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:49.392137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:49.392137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:49.392137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:49.392137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:49.392137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:49.392137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:49.392137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:49.392839 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 205/405] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:53:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:53:49.404449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75085c0-a755-4daf-9e48-b3cc5d8ac6f4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:49.415103 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:49.457704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd971b-68e6-4dc1-94d4-0e2d459a4055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:49.458367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd971b-68e6-4dc1-94d4-0e2d459a4055 None admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:49.458640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd971b-68e6-4dc1-94d4-0e2d459a4055 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:49.458905 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd971b-68e6-4dc1-94d4-0e2d459a4055 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551/groups/9952ba9059344824b23210e7f3d962ab/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:49.462746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd971b-68e6-4dc1-94d4-0e2d459a4055 None admin] RBAC: Authorizing `identity:create_grant(project_id=39c302b22dd04479bf8305109d613551, group_id=9952ba9059344824b23210e7f3d962ab, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:49.464232 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd971b-68e6-4dc1-94d4-0e2d459a4055 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:49.551694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75085c0-a755-4daf-9e48-b3cc5d8ac6f4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:49.569821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 201/406] 10.222.0.30 () {58 vars in 1442 bytes} [Thu Oct 6 07:53:49 2022] PUT /identity/v3/projects/39c302b22dd04479bf8305109d613551/groups/9952ba9059344824b23210e7f3d962ab/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:53:49.571463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd971b-68e6-4dc1-94d4-0e2d459a4055 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:49.623901 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdb812-0b7b-4900-b4b8-49891339700b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:49.625689 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdb812-0b7b-4900-b4b8-49891339700b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:49.626232 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdb812-0b7b-4900-b4b8-49891339700b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:49.627286 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdb812-0b7b-4900-b4b8-49891339700b None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:49.628423 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdb812-0b7b-4900-b4b8-49891339700b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:49.629131 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdb812-0b7b-4900-b4b8-49891339700b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:49.631827 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdb812-0b7b-4900-b4b8-49891339700b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:49.654407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 206/407] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:53:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:49.661604 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdb812-0b7b-4900-b4b8-49891339700b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:49.707384 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:49.746814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd971b-68e6-4dc1-94d4-0e2d459a4055 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:49.804144 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:49.823581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6562e4-5d71-4be0-b4de-05cd22e08ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:49.824903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6562e4-5d71-4be0-b4de-05cd22e08ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:49.825374 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6562e4-5d71-4be0-b4de-05cd22e08ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:49.825832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6562e4-5d71-4be0-b4de-05cd22e08ca4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:49.838765 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6562e4-5d71-4be0-b4de-05cd22e08ca4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:49.846462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-7c6562e4-5d71-4be0-b4de-05cd22e08ca4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:49.846462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:49.846462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:49.846462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:49.846462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:49.846462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:49.846462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:49.846462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:49.846462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:49.846462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:49.846462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:49.846462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:49.846462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:49.846462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:49.846462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:49.846462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:49.847393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:49.847393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:49.847393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:49.847393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:49.847393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:49.847393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:49.847393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:49.847393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:49.847393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:49.847393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:49.847393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:49.847393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:49.847393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:49.847393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:49.847393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:49.847393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:49.848184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:49.848184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:49.848184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:49.848184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:49.848184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:49.848184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:49.848184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:49.848184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:49.848184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:49.848184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:49.848184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:49.848184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:49.850704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 202/408] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:53:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:49.855032 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdb812-0b7b-4900-b4b8-49891339700b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:49.857472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6562e4-5d71-4be0-b4de-05cd22e08ca4 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:49.907785 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08dcbb-eddf-46dd-bb5f-f857f7ef3377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:49.912875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08dcbb-eddf-46dd-bb5f-f857f7ef3377 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:49.913364 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08dcbb-eddf-46dd-bb5f-f857f7ef3377 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:49.913880 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08dcbb-eddf-46dd-bb5f-f857f7ef3377 None admin] PATH_INFO: `/v3/groups/9952ba9059344824b23210e7f3d962ab` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:49.915286 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b08dcbb-eddf-46dd-bb5f-f857f7ef3377 None admin] RBAC: Authorizing `identity:get_group(group_id=9952ba9059344824b23210e7f3d962ab)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:49.916925 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b08dcbb-eddf-46dd-bb5f-f857f7ef3377 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:49.918703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 207/409] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Oct 6 07:53:49 2022] GET /identity/v3/groups/9952ba9059344824b23210e7f3d962ab => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:49.924363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08dcbb-eddf-46dd-bb5f-f857f7ef3377 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:50.035368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6562e4-5d71-4be0-b4de-05cd22e08ca4 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:50.099236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:50.111116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a81ab1-1688-48f1-b3a6-0cdc91112706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:50.114110 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a81ab1-1688-48f1-b3a6-0cdc91112706 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:50.114110 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a81ab1-1688-48f1-b3a6-0cdc91112706 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:50.114110 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a81ab1-1688-48f1-b3a6-0cdc91112706 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:50.114110 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a81ab1-1688-48f1-b3a6-0cdc91112706 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:50.114110 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a81ab1-1688-48f1-b3a6-0cdc91112706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:50.114559 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a81ab1-1688-48f1-b3a6-0cdc91112706 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:50.124518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 203/410] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:50.130077 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a81ab1-1688-48f1-b3a6-0cdc91112706 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:50.136538 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08dcbb-eddf-46dd-bb5f-f857f7ef3377 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:50.194770 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83491cc5-f98e-4bc8-a7ac-572581fabf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:50.195466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83491cc5-f98e-4bc8-a7ac-572581fabf99 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:50.195821 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83491cc5-f98e-4bc8-a7ac-572581fabf99 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:50.196111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83491cc5-f98e-4bc8-a7ac-572581fabf99 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:50.198998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83491cc5-f98e-4bc8-a7ac-572581fabf99 None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:50.200611 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83491cc5-f98e-4bc8-a7ac-572581fabf99 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:50.202553 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 208/411] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:49 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:50.208105 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83491cc5-f98e-4bc8-a7ac-572581fabf99 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:50.217519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:50.327863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83491cc5-f98e-4bc8-a7ac-572581fabf99 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:50.345650 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a81ab1-1688-48f1-b3a6-0cdc91112706 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:50.372569 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a67af-f2d8-413b-aa98-db36d0937025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:50.373152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a67af-f2d8-413b-aa98-db36d0937025 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:50.373690 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a67af-f2d8-413b-aa98-db36d0937025 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:50.373957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a67af-f2d8-413b-aa98-db36d0937025 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551/groups/9952ba9059344824b23210e7f3d962ab/roles/d9b2237fd9884cc39771e75bf902e203` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:50.377003 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a67af-f2d8-413b-aa98-db36d0937025 None admin] RBAC: Authorizing `identity:create_grant(project_id=39c302b22dd04479bf8305109d613551, group_id=9952ba9059344824b23210e7f3d962ab, role_id=d9b2237fd9884cc39771e75bf902e203)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:50.378442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a67af-f2d8-413b-aa98-db36d0937025 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:50.402988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746015a3-8622-4e8e-82ab-28575dea327c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:50.403678 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746015a3-8622-4e8e-82ab-28575dea327c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:50.403961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746015a3-8622-4e8e-82ab-28575dea327c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:50.404231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746015a3-8622-4e8e-82ab-28575dea327c None admin] PATH_INFO: `/v3/groups/caa581df8bd54f6a86f3e0f8016b290c` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:50.405561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746015a3-8622-4e8e-82ab-28575dea327c None admin] RBAC: Authorizing `identity:get_group(group_id=caa581df8bd54f6a86f3e0f8016b290c)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:50.407055 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746015a3-8622-4e8e-82ab-28575dea327c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:50.409009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 204/412] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Oct 6 07:53:50 2022] GET /identity/v3/groups/caa581df8bd54f6a86f3e0f8016b290c => generated 229 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:50.415355 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746015a3-8622-4e8e-82ab-28575dea327c None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:50.464377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 209/413] 10.222.0.30 () {58 vars in 1442 bytes} [Thu Oct 6 07:53:50 2022] PUT /identity/v3/projects/39c302b22dd04479bf8305109d613551/groups/9952ba9059344824b23210e7f3d962ab/roles/d9b2237fd9884cc39771e75bf902e203 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:53:50.487682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:50.545147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746015a3-8622-4e8e-82ab-28575dea327c None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:50.585192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648d459-80c8-48a1-87dd-58502acd8718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:50.586783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648d459-80c8-48a1-87dd-58502acd8718 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:50.587335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648d459-80c8-48a1-87dd-58502acd8718 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:50.587950 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648d459-80c8-48a1-87dd-58502acd8718 None admin] PATH_INFO: `/v3/projects/f2d30f50f03441f1a79068b287cb95c1` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:50.589965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648d459-80c8-48a1-87dd-58502acd8718 None admin] RBAC: Authorizing `identity:get_project(project_id=f2d30f50f03441f1a79068b287cb95c1)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:50.592119 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648d459-80c8-48a1-87dd-58502acd8718 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:50.594473 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 205/414] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:50 2022] GET /identity/v3/projects/f2d30f50f03441f1a79068b287cb95c1 => generated 344 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:50.606432 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a67af-f2d8-413b-aa98-db36d0937025 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:50.683754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:50.727630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a67af-f2d8-413b-aa98-db36d0937025 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:50.763244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6be220-9f95-4afa-b1ed-2f52ac0fd8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:50.763907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6be220-9f95-4afa-b1ed-2f52ac0fd8db None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:50.764181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6be220-9f95-4afa-b1ed-2f52ac0fd8db None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:50.764452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6be220-9f95-4afa-b1ed-2f52ac0fd8db None admin] PATH_INFO: `/v3/projects/f2d30f50f03441f1a79068b287cb95c1/groups/caa581df8bd54f6a86f3e0f8016b290c/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:50.766364 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6be220-9f95-4afa-b1ed-2f52ac0fd8db None admin] RBAC: Authorizing `identity:create_grant(project_id=f2d30f50f03441f1a79068b287cb95c1, group_id=caa581df8bd54f6a86f3e0f8016b290c, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:50.767832 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6be220-9f95-4afa-b1ed-2f52ac0fd8db None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:50.825150 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 210/415] 10.222.0.30 () {58 vars in 1442 bytes} [Thu Oct 6 07:53:50 2022] PUT /identity/v3/projects/f2d30f50f03441f1a79068b287cb95c1/groups/caa581df8bd54f6a86f3e0f8016b290c/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:53:50.930492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09663b94-995f-4650-84f7-2300987ef5ee None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:50.931495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09663b94-995f-4650-84f7-2300987ef5ee None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:50.932026 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09663b94-995f-4650-84f7-2300987ef5ee None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:50.933931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 206/416] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:50.939319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970fcd4-9c20-4be6-9d54-fb385d608465 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:50.939595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970fcd4-9c20-4be6-9d54-fb385d608465 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:50.939865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970fcd4-9c20-4be6-9d54-fb385d608465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:50.968133 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e970fcd4-9c20-4be6-9d54-fb385d608465 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:50.969363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e970fcd4-9c20-4be6-9d54-fb385d608465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:51.068858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 211/417] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:51.188462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ae8ab-2a98-40b0-9d9f-405cfed89b45 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:51.189054 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ae8ab-2a98-40b0-9d9f-405cfed89b45 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:51.189532 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ae8ab-2a98-40b0-9d9f-405cfed89b45 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:51.191225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 207/418] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:51.196635 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbac44-7c8a-431b-ad5e-eb5c44328511 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:51.197176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbac44-7c8a-431b-ad5e-eb5c44328511 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:51.197666 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbac44-7c8a-431b-ad5e-eb5c44328511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:51.217404 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a2dbac44-7c8a-431b-ad5e-eb5c44328511 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:51.222962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a2dbac44-7c8a-431b-ad5e-eb5c44328511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:51.286798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:51.316095 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 212/419] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:51.376323 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ae8ab-2a98-40b0-9d9f-405cfed89b45 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:51.516591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ae8ab-2a98-40b0-9d9f-405cfed89b45 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:51.549437 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:51.557375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7dca85-a213-448b-99b7-c14699f53edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:51.558010 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7dca85-a213-448b-99b7-c14699f53edf None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:51.564805 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7dca85-a213-448b-99b7-c14699f53edf None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:51.564805 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7dca85-a213-448b-99b7-c14699f53edf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:51.575817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7dca85-a213-448b-99b7-c14699f53edf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:51.580682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-7b7dca85-a213-448b-99b7-c14699f53edf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:51.580682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:51.580682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:51.580682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:51.580682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:51.580682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:51.580682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:51.580682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:51.580682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:51.580682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:51.580682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:51.580682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:51.580682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:51.580682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:51.580682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:51.580682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:51.581605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:51.581605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:51.581605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:51.581605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:51.581605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:51.581605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:51.581605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:51.581605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:51.581605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:51.581605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:51.581605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:51.581605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:51.581605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:51.581605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:51.581605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:51.581605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:51.582591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:51.582591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:51.582591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:51.582591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:51.582591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:51.582591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:51.582591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:51.582591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:51.582591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:51.582591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:51.582591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:51.582591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:51.582591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 208/420] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:53:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:51.588897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbac44-7c8a-431b-ad5e-eb5c44328511 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:51.715506 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbac44-7c8a-431b-ad5e-eb5c44328511 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:51.756088 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723c961-662d-4ae9-ac0d-f0c7fd7b3f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:51.756697 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723c961-662d-4ae9-ac0d-f0c7fd7b3f90 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:51.756977 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723c961-662d-4ae9-ac0d-f0c7fd7b3f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:51.757759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723c961-662d-4ae9-ac0d-f0c7fd7b3f90 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:51.758446 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723c961-662d-4ae9-ac0d-f0c7fd7b3f90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:51.758749 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723c961-662d-4ae9-ac0d-f0c7fd7b3f90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:51.760149 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723c961-662d-4ae9-ac0d-f0c7fd7b3f90 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:51.768976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 213/421] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:53:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:51.780786 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7dca85-a213-448b-99b7-c14699f53edf None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:51.916359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723c961-662d-4ae9-ac0d-f0c7fd7b3f90 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:51.919041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7dca85-a213-448b-99b7-c14699f53edf None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:51.963647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b4f5f-bca2-4d5a-a61c-1f7030493bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:51.964367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b4f5f-bca2-4d5a-a61c-1f7030493bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:51.964669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b4f5f-bca2-4d5a-a61c-1f7030493bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:51.964942 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b4f5f-bca2-4d5a-a61c-1f7030493bd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:51.968908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:52.024334 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:52.036865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b4f5f-bca2-4d5a-a61c-1f7030493bd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:52.038350 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b4f5f-bca2-4d5a-a61c-1f7030493bd7 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:52.065674 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723c961-662d-4ae9-ac0d-f0c7fd7b3f90 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:52.127366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14808d-0280-429c-acc0-4c1c655bbfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:52.128494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14808d-0280-429c-acc0-4c1c655bbfd1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:52.129856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14808d-0280-429c-acc0-4c1c655bbfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:52.131140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14808d-0280-429c-acc0-4c1c655bbfd1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:52.152601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14808d-0280-429c-acc0-4c1c655bbfd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:52.167042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-8d14808d-0280-429c-acc0-4c1c655bbfd1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:52.167042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:52.167042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:52.167042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:52.167042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:52.167042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:52.167042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:52.167042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:52.167042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:52.167042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:52.167042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:52.167042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:52.167042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:52.167042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:52.167042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:52.167042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:52.167966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:52.167966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:52.167966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:52.167966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:52.167966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:52.167966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:52.167966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:52.167966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:52.167966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:52.167966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:52.167966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:52.167966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:52.167966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:52.167966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:52.167966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:52.167966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:52.168754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:52.168754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:52.168754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:52.168754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:52.168754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:52.168754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:52.168754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:52.168754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:52.168754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:52.168754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:52.168754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:52.168754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:52.174407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 214/422] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:53:52.175878 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14808d-0280-429c-acc0-4c1c655bbfd1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:52.245073 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 209/423] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:52.250833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b4f5f-bca2-4d5a-a61c-1f7030493bd7 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:52.353678 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14808d-0280-429c-acc0-4c1c655bbfd1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:52.377069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:52.421371 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab860fa-d921-4170-95d7-4435056ce345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:52.421982 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab860fa-d921-4170-95d7-4435056ce345 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:52.422257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab860fa-d921-4170-95d7-4435056ce345 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:52.422527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab860fa-d921-4170-95d7-4435056ce345 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:52.423234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab860fa-d921-4170-95d7-4435056ce345 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:52.423543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab860fa-d921-4170-95d7-4435056ce345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:52.424913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab860fa-d921-4170-95d7-4435056ce345 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:52.431323 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:52.447055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 215/424] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:52.448521 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab860fa-d921-4170-95d7-4435056ce345 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:52.455392 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b4f5f-bca2-4d5a-a61c-1f7030493bd7 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:52.497100 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07227c18-23d5-41e2-b192-a9292051f7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:52.498196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07227c18-23d5-41e2-b192-a9292051f7bb None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:52.498922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07227c18-23d5-41e2-b192-a9292051f7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:52.499438 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07227c18-23d5-41e2-b192-a9292051f7bb None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:52.500775 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07227c18-23d5-41e2-b192-a9292051f7bb None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:52.502412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07227c18-23d5-41e2-b192-a9292051f7bb None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:52.504660 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 210/425] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:52 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:52.505984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a019755-70e7-4df8-95d4-8be2d3ea87bb None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:52.506460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a019755-70e7-4df8-95d4-8be2d3ea87bb None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:52.506920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a019755-70e7-4df8-95d4-8be2d3ea87bb None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:52.512728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 211/426] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:52.514438 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca5ae1-1900-4156-b2fe-d95f376c2bb2 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:52.515052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca5ae1-1900-4156-b2fe-d95f376c2bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:52.515519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca5ae1-1900-4156-b2fe-d95f376c2bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:52.534808 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-e7ca5ae1-1900-4156-b2fe-d95f376c2bb2 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:52.536462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e7ca5ae1-1900-4156-b2fe-d95f376c2bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:52.570346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab860fa-d921-4170-95d7-4435056ce345 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:52.605548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907162a-f57e-49cb-9ba3-b25e578c337c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:52.606135 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907162a-f57e-49cb-9ba3-b25e578c337c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:52.606338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907162a-f57e-49cb-9ba3-b25e578c337c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:52.606619 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907162a-f57e-49cb-9ba3-b25e578c337c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:52.615181 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 212/427] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:53:52.616388 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca5ae1-1900-4156-b2fe-d95f376c2bb2 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:52.644897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:52.682262 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907162a-f57e-49cb-9ba3-b25e578c337c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:52.683756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907162a-f57e-49cb-9ba3-b25e578c337c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:52.793711 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 216/428] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 06 07:53:52.794983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907162a-f57e-49cb-9ba3-b25e578c337c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:52.863449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:52.868153 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:52.901122 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907162a-f57e-49cb-9ba3-b25e578c337c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:52.910665 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca5ae1-1900-4156-b2fe-d95f376c2bb2 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:52.940148 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a84074-d562-4711-8c10-88389b90e0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:52.940908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a84074-d562-4711-8c10-88389b90e0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:52.941255 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a84074-d562-4711-8c10-88389b90e0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:52.941605 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a84074-d562-4711-8c10-88389b90e0e7 None admin] PATH_INFO: `/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:52.942819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a84074-d562-4711-8c10-88389b90e0e7 None admin] RBAC: Authorizing `identity:get_project(project_id=e85a26a4bf0d46a29c6c84d27f4fbd45)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:52.944534 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a84074-d562-4711-8c10-88389b90e0e7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:52.951971 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 217/429] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:52 2022] GET /identity/v3/projects/e85a26a4bf0d46a29c6c84d27f4fbd45 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:52.953217 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a84074-d562-4711-8c10-88389b90e0e7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:52.998274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a453ccd-82a5-493b-b342-1bce05da6c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:52.999255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a453ccd-82a5-493b-b342-1bce05da6c84 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:52.999538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a453ccd-82a5-493b-b342-1bce05da6c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:52.999811 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a453ccd-82a5-493b-b342-1bce05da6c84 None admin] PATH_INFO: `/v3/groups/9952ba9059344824b23210e7f3d962ab` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:53.000951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a453ccd-82a5-493b-b342-1bce05da6c84 None admin] RBAC: Authorizing `identity:get_group(group_id=9952ba9059344824b23210e7f3d962ab)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:53.002441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a453ccd-82a5-493b-b342-1bce05da6c84 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:53.013941 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 213/430] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Oct 6 07:53:52 2022] GET /identity/v3/groups/9952ba9059344824b23210e7f3d962ab => generated 226 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:53.025127 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a453ccd-82a5-493b-b342-1bce05da6c84 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:53.078920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a84074-d562-4711-8c10-88389b90e0e7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:53.113798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f561e57-e0fa-42b5-b28e-d290d2cda07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:53.114356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f561e57-e0fa-42b5-b28e-d290d2cda07a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:53.114632 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f561e57-e0fa-42b5-b28e-d290d2cda07a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:53.114908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f561e57-e0fa-42b5-b28e-d290d2cda07a None admin] PATH_INFO: `/v3/groups/9952ba9059344824b23210e7f3d962ab` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:53.116054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f561e57-e0fa-42b5-b28e-d290d2cda07a None admin] RBAC: Authorizing `identity:get_group(group_id=9952ba9059344824b23210e7f3d962ab)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:53.117530 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f561e57-e0fa-42b5-b28e-d290d2cda07a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:53.119280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 218/431] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Oct 6 07:53:52 2022] GET /identity/v3/groups/9952ba9059344824b23210e7f3d962ab => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:53.120380 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b58d5-3104-4f7e-8637-268997792e87 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:53.120665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b58d5-3104-4f7e-8637-268997792e87 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:53.120937 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b58d5-3104-4f7e-8637-268997792e87 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:53.126449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 219/432] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:53.126449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b58d5-3104-4f7e-8637-268997792e87 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:53.128778 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:53.139742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a453ccd-82a5-493b-b342-1bce05da6c84 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:53.180636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a7975-801a-4310-a01b-7fa7a4b7eef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:53.181246 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a7975-801a-4310-a01b-7fa7a4b7eef1 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:53.181523 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a7975-801a-4310-a01b-7fa7a4b7eef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:53.181797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a7975-801a-4310-a01b-7fa7a4b7eef1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:53.182527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a7975-801a-4310-a01b-7fa7a4b7eef1 None admin] RBAC: Adding query filter params (group.id=9952ba9059344824b23210e7f3d962ab, role.id=d9b2237fd9884cc39771e75bf902e203, scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:53.182832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a7975-801a-4310-a01b-7fa7a4b7eef1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:53.184322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a7975-801a-4310-a01b-7fa7a4b7eef1 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:53.191340 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 214/433] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Oct 6 07:53:53 2022] GET /identity/v3/role_assignments?group.id=9952ba9059344824b23210e7f3d962ab&scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45&role.id=d9b2237fd9884cc39771e75bf902e203 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:53.192153 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a7975-801a-4310-a01b-7fa7a4b7eef1 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:53.220082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:53.259176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b58d5-3104-4f7e-8637-268997792e87 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:53.298466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0943ef-a7c0-484c-a2b3-38e7bc88f570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:53.299148 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0943ef-a7c0-484c-a2b3-38e7bc88f570 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:53.299428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0943ef-a7c0-484c-a2b3-38e7bc88f570 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:53.299709 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0943ef-a7c0-484c-a2b3-38e7bc88f570 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:53.312551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0943ef-a7c0-484c-a2b3-38e7bc88f570 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:53.317589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-9a0943ef-a7c0-484c-a2b3-38e7bc88f570 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:53.317589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:53.317589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:53.317589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:53.317589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:53.317589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:53.317589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:53.317589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:53.317589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:53.317589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:53.317589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:53.317589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:53.317589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:53.317589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:53.317589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:53.317589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:53.318544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:53.318544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:53.318544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:53.318544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:53.318544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:53.318544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:53.318544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:53.318544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:53.318544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:53.318544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:53.318544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:53.318544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:53.318544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:53.318544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:53.318544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:53.318544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:53.319374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:53.319374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:53.319374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:53.319374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:53.319374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:53.319374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:53.319374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:53.319374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:53.319374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:53.319374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:53.319374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 06 07:53:53.319374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:53.323373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 220/434] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:53.325340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23467307-edb9-4d75-9f04-b0f406b586f4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:53.325625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23467307-edb9-4d75-9f04-b0f406b586f4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:53.325899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23467307-edb9-4d75-9f04-b0f406b586f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:53.332506 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a7975-801a-4310-a01b-7fa7a4b7eef1 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:53.346319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-23467307-edb9-4d75-9f04-b0f406b586f4 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:53.347588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-23467307-edb9-4d75-9f04-b0f406b586f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:53.373948 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78523ae-9d40-4f9a-96b8-500c11020f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:53.374994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78523ae-9d40-4f9a-96b8-500c11020f74 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:53.374994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78523ae-9d40-4f9a-96b8-500c11020f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:53.375572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78523ae-9d40-4f9a-96b8-500c11020f74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:53.376320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78523ae-9d40-4f9a-96b8-500c11020f74 None admin] RBAC: Adding query filter params (group.id=9952ba9059344824b23210e7f3d962ab, role.id=dd5cafa908a74cfd8e1a8baa44122242, scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:53.376635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78523ae-9d40-4f9a-96b8-500c11020f74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:53.378236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78523ae-9d40-4f9a-96b8-500c11020f74 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:53.386156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 215/435] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Oct 6 07:53:53 2022] GET /identity/v3/role_assignments?group.id=9952ba9059344824b23210e7f3d962ab&scope.project.id=e85a26a4bf0d46a29c6c84d27f4fbd45&role.id=dd5cafa908a74cfd8e1a8baa44122242 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:53.386656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78523ae-9d40-4f9a-96b8-500c11020f74 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:53.415891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:53.430597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 221/436] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:53.432502 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc89a8f-c83a-4994-b20c-39957274cf06 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:53.432850 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc89a8f-c83a-4994-b20c-39957274cf06 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:53.433197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc89a8f-c83a-4994-b20c-39957274cf06 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:53.435004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 222/437] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:53.440275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67885b8-d4eb-4082-bada-d18b0acf9a98 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:53.440606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67885b8-d4eb-4082-bada-d18b0acf9a98 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:53.440883 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67885b8-d4eb-4082-bada-d18b0acf9a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:53.468129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c67885b8-d4eb-4082-bada-d18b0acf9a98 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:53.469373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c67885b8-d4eb-4082-bada-d18b0acf9a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:53.503340 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:53.525507 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78523ae-9d40-4f9a-96b8-500c11020f74 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:53.553022 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 223/438] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:53.571696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2c6b5-2083-40b5-9893-f398d12a48bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:53.572565 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2c6b5-2083-40b5-9893-f398d12a48bf None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:53.573020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2c6b5-2083-40b5-9893-f398d12a48bf None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:53.573441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2c6b5-2083-40b5-9893-f398d12a48bf None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:53.574267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2c6b5-2083-40b5-9893-f398d12a48bf None admin] RBAC: Adding query filter params (name=member) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:53.574715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2c6b5-2083-40b5-9893-f398d12a48bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:53.576325 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2c6b5-2083-40b5-9893-f398d12a48bf None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:53.587101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 216/439] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:53:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:53.592808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67885b8-d4eb-4082-bada-d18b0acf9a98 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:53.702385 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67885b8-d4eb-4082-bada-d18b0acf9a98 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:53.727329 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:53.732400 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2c6b5-2083-40b5-9893-f398d12a48bf None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:53.740496 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d64da-6426-4e13-a278-e57ea617850b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:53.741115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d64da-6426-4e13-a278-e57ea617850b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:53.741392 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d64da-6426-4e13-a278-e57ea617850b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:53.742279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d64da-6426-4e13-a278-e57ea617850b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:53.835599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d64da-6426-4e13-a278-e57ea617850b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:53.837269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d64da-6426-4e13-a278-e57ea617850b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:53.863051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2c6b5-2083-40b5-9893-f398d12a48bf None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:53.901478 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50620ae9-1d9b-4c16-b368-0434b30ab67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:53.903690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50620ae9-1d9b-4c16-b368-0434b30ab67e None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:53.903690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50620ae9-1d9b-4c16-b368-0434b30ab67e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:53.903690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50620ae9-1d9b-4c16-b368-0434b30ab67e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:53.916911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50620ae9-1d9b-4c16-b368-0434b30ab67e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:53.922101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-50620ae9-1d9b-4c16-b368-0434b30ab67e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:53.922101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:53.922101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:53.922101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:53.922101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:53.922101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:53.922101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:53.922101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:53.922101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:53.922101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:53.922101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:53.922101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:53.922101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:53.922101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:53.922101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:53.922101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:53.923097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:53.923097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:53.923097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:53.923097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:53.923097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:53.923097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:53.923097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:53.923097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:53.923097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:53.923097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:53.923097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:53.923097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:53.923097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:53.923097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:53.923097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:53.923097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:53.923916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:53.923916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:53.923916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:53.923916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:53.923916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:53.923916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:53.923916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:53.923916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:53.923916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:53.923916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:53.923916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:53:53.923916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:53.923916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 217/440] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:53:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:53:53.924577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50620ae9-1d9b-4c16-b368-0434b30ab67e None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:53.927867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:53.955094 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 224/441] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:53.955916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d64da-6426-4e13-a278-e57ea617850b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:53.986500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:54.045545 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50620ae9-1d9b-4c16-b368-0434b30ab67e None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:54.073965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d64da-6426-4e13-a278-e57ea617850b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:54.079608 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3e126-85e3-452f-a5f7-63684752fb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:54.080226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3e126-85e3-452f-a5f7-63684752fb35 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:54.080501 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3e126-85e3-452f-a5f7-63684752fb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:54.080770 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3e126-85e3-452f-a5f7-63684752fb35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:54.091671 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3e126-85e3-452f-a5f7-63684752fb35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:53:54.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-acd3e126-85e3-452f-a5f7-63684752fb35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:54.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:54.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:54.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:54.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:54.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:54.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:54.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:54.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:54.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:54.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:54.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:54.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:53:54.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:53:54.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:53:54.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:53:54.097909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:53:54.097909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:53:54.097909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:53:54.097909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:53:54.097909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:54.097909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:54.097909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:54.097909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:54.097909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:54.097909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:54.097909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:54.097909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:54.097909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:54.097909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:54.097909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:54.097909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:54.098902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:54.098902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:54.098902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:54.098902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:54.098902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:53:54.098902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:53:54.098902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:53:54.098902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:53:54.098902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:53:54.098902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:53:54.098902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:53:54.098902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:54.098902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 218/442] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:53:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:54.098902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3e126-85e3-452f-a5f7-63684752fb35 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:54.112210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e81a4-d09c-4d04-acad-a60edbc574ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:54.112818 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e81a4-d09c-4d04-acad-a60edbc574ec None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:54.113095 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e81a4-d09c-4d04-acad-a60edbc574ec None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:54.113373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e81a4-d09c-4d04-acad-a60edbc574ec None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:54.114035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e81a4-d09c-4d04-acad-a60edbc574ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:54.114341 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e81a4-d09c-4d04-acad-a60edbc574ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:54.115746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e81a4-d09c-4d04-acad-a60edbc574ec None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:54.124404 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 225/443] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:53:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:54.125044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e81a4-d09c-4d04-acad-a60edbc574ec None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:54.164373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:54.202340 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3e126-85e3-452f-a5f7-63684752fb35 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:54.213231 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:54.228660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e81a4-d09c-4d04-acad-a60edbc574ec None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:54.238489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b1f3f-ebc6-4f72-97f6-471f1c337d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:54.239151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b1f3f-ebc6-4f72-97f6-471f1c337d7b None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:54.239431 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b1f3f-ebc6-4f72-97f6-471f1c337d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:54.239706 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b1f3f-ebc6-4f72-97f6-471f1c337d7b None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:54.240843 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b1f3f-ebc6-4f72-97f6-471f1c337d7b None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:54.242828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b1f3f-ebc6-4f72-97f6-471f1c337d7b None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:54.244534 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 219/444] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:54 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:54.245298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b1f3f-ebc6-4f72-97f6-471f1c337d7b None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:54.275201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd84476-fb33-478f-8fcf-12ec32294340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:54.275820 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd84476-fb33-478f-8fcf-12ec32294340 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:54.276096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd84476-fb33-478f-8fcf-12ec32294340 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:54.276366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd84476-fb33-478f-8fcf-12ec32294340 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:54.277034 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd84476-fb33-478f-8fcf-12ec32294340 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:54.277333 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd84476-fb33-478f-8fcf-12ec32294340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:54.278699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd84476-fb33-478f-8fcf-12ec32294340 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:54.287035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 226/445] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:53:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:54.287837 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd84476-fb33-478f-8fcf-12ec32294340 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:54.348938 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b1f3f-ebc6-4f72-97f6-471f1c337d7b None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:54.377741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:54.383266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099e7eb-27fe-40ff-9e3c-e9dcd0bbc0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:54.383840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099e7eb-27fe-40ff-9e3c-e9dcd0bbc0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:54.384117 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099e7eb-27fe-40ff-9e3c-e9dcd0bbc0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:54.384389 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099e7eb-27fe-40ff-9e3c-e9dcd0bbc0f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:54.389906 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd84476-fb33-478f-8fcf-12ec32294340 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:54.422287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e38fb5-84c7-4616-abf3-790d306aefb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:54.422905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e38fb5-84c7-4616-abf3-790d306aefb0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:54.423211 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e38fb5-84c7-4616-abf3-790d306aefb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:54.423479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e38fb5-84c7-4616-abf3-790d306aefb0 None admin] PATH_INFO: `/v3/groups/9952ba9059344824b23210e7f3d962ab` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:54.424589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e38fb5-84c7-4616-abf3-790d306aefb0 None admin] RBAC: Authorizing `identity:get_group(group_id=9952ba9059344824b23210e7f3d962ab)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:54.426021 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e38fb5-84c7-4616-abf3-790d306aefb0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:54.427613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 227/446] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Oct 6 07:53:54 2022] GET /identity/v3/groups/9952ba9059344824b23210e7f3d962ab => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:54.428366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e38fb5-84c7-4616-abf3-790d306aefb0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:54.446647 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:54.451657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099e7eb-27fe-40ff-9e3c-e9dcd0bbc0f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:54.453121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099e7eb-27fe-40ff-9e3c-e9dcd0bbc0f5 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:54.530386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e38fb5-84c7-4616-abf3-790d306aefb0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:54.550257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 220/447] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:54.551125 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099e7eb-27fe-40ff-9e3c-e9dcd0bbc0f5 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:54.567435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a96fa-7e5f-493c-be65-c40cebf5158e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:54.568039 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a96fa-7e5f-493c-be65-c40cebf5158e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:54.568311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a96fa-7e5f-493c-be65-c40cebf5158e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:54.568586 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a96fa-7e5f-493c-be65-c40cebf5158e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:54.580518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:54.636197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a96fa-7e5f-493c-be65-c40cebf5158e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:54.637669 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a96fa-7e5f-493c-be65-c40cebf5158e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:54.649209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099e7eb-27fe-40ff-9e3c-e9dcd0bbc0f5 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:54.652925 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:54.681967 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfda411-05af-41e2-8612-4989853960ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:54.682564 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfda411-05af-41e2-8612-4989853960ea None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:54.682840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfda411-05af-41e2-8612-4989853960ea None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:54.683148 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfda411-05af-41e2-8612-4989853960ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:54.683870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfda411-05af-41e2-8612-4989853960ea None admin] RBAC: Adding query filter params (group.id=9952ba9059344824b23210e7f3d962ab, role.id=dd5cafa908a74cfd8e1a8baa44122242, scope.project.id=39c302b22dd04479bf8305109d613551) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:54.684178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfda411-05af-41e2-8612-4989853960ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:54.685616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfda411-05af-41e2-8612-4989853960ea None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:54.691411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 221/448] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Oct 6 07:53:54 2022] GET /identity/v3/role_assignments?group.id=9952ba9059344824b23210e7f3d962ab&scope.project.id=39c302b22dd04479bf8305109d613551&role.id=dd5cafa908a74cfd8e1a8baa44122242 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:54.692104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfda411-05af-41e2-8612-4989853960ea None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:54.763219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 228/449] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:53:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:53:54.769127 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a96fa-7e5f-493c-be65-c40cebf5158e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:54.803661 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:54.810828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfda411-05af-41e2-8612-4989853960ea None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:54.848647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cceefe0-6763-478e-bbe7-5f6e827ed1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:54.849296 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cceefe0-6763-478e-bbe7-5f6e827ed1d0 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:54.849576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cceefe0-6763-478e-bbe7-5f6e827ed1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:54.849850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cceefe0-6763-478e-bbe7-5f6e827ed1d0 None admin] PATH_INFO: `/v3/projects/39c302b22dd04479bf8305109d613551` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:54.852876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cceefe0-6763-478e-bbe7-5f6e827ed1d0 None admin] RBAC: Authorizing `identity:get_project(project_id=39c302b22dd04479bf8305109d613551)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:54.852876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cceefe0-6763-478e-bbe7-5f6e827ed1d0 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:54.854545 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 222/450] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:54 2022] GET /identity/v3/projects/39c302b22dd04479bf8305109d613551 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:54.860109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cceefe0-6763-478e-bbe7-5f6e827ed1d0 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:54.890456 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a96fa-7e5f-493c-be65-c40cebf5158e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:54.908340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:54.930744 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4c186-bcd0-40f2-9f0e-1741cbee5d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:54.931452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4c186-bcd0-40f2-9f0e-1741cbee5d74 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:54.931725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4c186-bcd0-40f2-9f0e-1741cbee5d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:54.931992 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4c186-bcd0-40f2-9f0e-1741cbee5d74 None admin] PATH_INFO: `/v3/projects/f2d30f50f03441f1a79068b287cb95c1` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:54.933109 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4c186-bcd0-40f2-9f0e-1741cbee5d74 None admin] RBAC: Authorizing `identity:get_project(project_id=f2d30f50f03441f1a79068b287cb95c1)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:54.934599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4c186-bcd0-40f2-9f0e-1741cbee5d74 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:54.936876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 229/451] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:53:54 2022] GET /identity/v3/projects/f2d30f50f03441f1a79068b287cb95c1 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:54.942704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4c186-bcd0-40f2-9f0e-1741cbee5d74 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:54.982830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cceefe0-6763-478e-bbe7-5f6e827ed1d0 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:55.023867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a9c68-b56f-477b-a889-84abb997e80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:55.024496 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a9c68-b56f-477b-a889-84abb997e80d None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:55.025668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a9c68-b56f-477b-a889-84abb997e80d None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:55.025668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a9c68-b56f-477b-a889-84abb997e80d None admin] PATH_INFO: `/v3/groups/9952ba9059344824b23210e7f3d962ab` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:55.026193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a9c68-b56f-477b-a889-84abb997e80d None admin] RBAC: Authorizing `identity:get_group(group_id=9952ba9059344824b23210e7f3d962ab)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:55.027707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a9c68-b56f-477b-a889-84abb997e80d None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:55.029578 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 223/452] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Oct 6 07:53:54 2022] GET /identity/v3/groups/9952ba9059344824b23210e7f3d962ab => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:55.035399 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a9c68-b56f-477b-a889-84abb997e80d None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:55.064858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4c186-bcd0-40f2-9f0e-1741cbee5d74 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:55.073051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:55.101728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2577b3b-27ad-40c2-a9ab-42518c8dcf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:55.102357 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2577b3b-27ad-40c2-a9ab-42518c8dcf89 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:55.102632 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2577b3b-27ad-40c2-a9ab-42518c8dcf89 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:55.102903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2577b3b-27ad-40c2-a9ab-42518c8dcf89 None admin] PATH_INFO: `/v3/groups/caa581df8bd54f6a86f3e0f8016b290c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:55.104045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2577b3b-27ad-40c2-a9ab-42518c8dcf89 None admin] RBAC: Authorizing `identity:get_group(group_id=caa581df8bd54f6a86f3e0f8016b290c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:55.105518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2577b3b-27ad-40c2-a9ab-42518c8dcf89 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:55.107103 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 230/453] 10.222.0.30 () {56 vars in 1180 bytes} [Thu Oct 6 07:53:54 2022] GET /identity/v3/groups/caa581df8bd54f6a86f3e0f8016b290c => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:55.112769 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2577b3b-27ad-40c2-a9ab-42518c8dcf89 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:55.145989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a9c68-b56f-477b-a889-84abb997e80d None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:55.161372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:55.179946 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe0444-6ce6-47ae-afd7-f27bb6bf8b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:55.180542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe0444-6ce6-47ae-afd7-f27bb6bf8b89 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:55.180814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe0444-6ce6-47ae-afd7-f27bb6bf8b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:55.181082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe0444-6ce6-47ae-afd7-f27bb6bf8b89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:55.181802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe0444-6ce6-47ae-afd7-f27bb6bf8b89 None admin] RBAC: Adding query filter params (group.id=9952ba9059344824b23210e7f3d962ab, role.id=d9b2237fd9884cc39771e75bf902e203, scope.project.id=39c302b22dd04479bf8305109d613551) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:55.182100 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe0444-6ce6-47ae-afd7-f27bb6bf8b89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:55.183595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe0444-6ce6-47ae-afd7-f27bb6bf8b89 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:55.189358 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 224/454] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Oct 6 07:53:55 2022] GET /identity/v3/role_assignments?group.id=9952ba9059344824b23210e7f3d962ab&scope.project.id=39c302b22dd04479bf8305109d613551&role.id=d9b2237fd9884cc39771e75bf902e203 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:55.214340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2577b3b-27ad-40c2-a9ab-42518c8dcf89 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:55.250264 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb1d07-6c0c-4580-916c-e3f3242f46d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:55.250887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb1d07-6c0c-4580-916c-e3f3242f46d2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:55.251192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb1d07-6c0c-4580-916c-e3f3242f46d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:55.251465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb1d07-6c0c-4580-916c-e3f3242f46d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:55.252208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb1d07-6c0c-4580-916c-e3f3242f46d2 None admin] RBAC: Adding query filter params (group.id=caa581df8bd54f6a86f3e0f8016b290c, role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=f2d30f50f03441f1a79068b287cb95c1) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:55.252513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb1d07-6c0c-4580-916c-e3f3242f46d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:55.253976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb1d07-6c0c-4580-916c-e3f3242f46d2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:55.261310 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 231/455] 10.222.0.30 () {56 vars in 1376 bytes} [Thu Oct 6 07:53:55 2022] GET /identity/v3/role_assignments?group.id=caa581df8bd54f6a86f3e0f8016b290c&scope.project.id=f2d30f50f03441f1a79068b287cb95c1&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:53:57.890606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5683c-eb59-489d-8522-ed7865fd41c8 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:57.891654 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5683c-eb59-489d-8522-ed7865fd41c8 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:57.892161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5683c-eb59-489d-8522-ed7865fd41c8 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:57.893988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 225/456] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:57.904374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c3b8b-a3ac-4e99-be29-3f87dfb6fbb7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:57.905403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c3b8b-a3ac-4e99-be29-3f87dfb6fbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:57.905895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c3b8b-a3ac-4e99-be29-3f87dfb6fbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:57.932868 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-af3c3b8b-a3ac-4e99-be29-3f87dfb6fbb7 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:57.934417 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-af3c3b8b-a3ac-4e99-be29-3f87dfb6fbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:58.021567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 232/457] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:58.139617 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2d6cd-1d3f-49eb-94df-c39be5b07fac None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:58.139938 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2d6cd-1d3f-49eb-94df-c39be5b07fac None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:58.140159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2d6cd-1d3f-49eb-94df-c39be5b07fac None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:58.141813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 226/458] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:58.149633 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a6254-554b-468f-9481-1ae2c57a74a3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:58.149915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a6254-554b-468f-9481-1ae2c57a74a3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:58.150192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a6254-554b-468f-9481-1ae2c57a74a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:58.172376 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-165a6254-554b-468f-9481-1ae2c57a74a3 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:58.173886 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-165a6254-554b-468f-9481-1ae2c57a74a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:58.227019 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:58.269549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 233/459] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:58.467650 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2d6cd-1d3f-49eb-94df-c39be5b07fac None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:58.535952 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f6a43-1c46-4bde-b9e5-47390d1f5591 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:58.536270 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f6a43-1c46-4bde-b9e5-47390d1f5591 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:58.536794 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f6a43-1c46-4bde-b9e5-47390d1f5591 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:58.538177 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 234/460] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:58.543293 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86089b-2b96-4283-bf85-8af59fe4a22f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:58.543557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86089b-2b96-4283-bf85-8af59fe4a22f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:58.543832 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86089b-2b96-4283-bf85-8af59fe4a22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:58.576521 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ab86089b-2b96-4283-bf85-8af59fe4a22f None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:58.577852 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ab86089b-2b96-4283-bf85-8af59fe4a22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:58.592431 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:58.604652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2d6cd-1d3f-49eb-94df-c39be5b07fac None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:58.675269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 235/461] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:58.685690 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86089b-2b96-4283-bf85-8af59fe4a22f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:58.686166 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95d7a1-722a-421b-9a80-7a65f79b7df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:58.686803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95d7a1-722a-421b-9a80-7a65f79b7df2 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:58.687179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95d7a1-722a-421b-9a80-7a65f79b7df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:58.687447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95d7a1-722a-421b-9a80-7a65f79b7df2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:58.709999 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95d7a1-722a-421b-9a80-7a65f79b7df2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:58.712074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95d7a1-722a-421b-9a80-7a65f79b7df2 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:58.740619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-0a95d7a1-722a-421b-9a80-7a65f79b7df2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:53:58.740619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:58.740619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:58.740619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:58.740619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:58.740619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:58.740619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:58.740619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:58.740619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:58.740619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:58.740619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:58.740619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:58.740619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:53:58.740619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:53:58.740619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:53:58.740619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:58.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:53:58.742459 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:53:58.742459 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:53:58.742459 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:58.742459 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 227/462] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:58.748977 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95d7a1-722a-421b-9a80-7a65f79b7df2 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:58.870454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86089b-2b96-4283-bf85-8af59fe4a22f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:58.891325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:58.891754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95d7a1-722a-421b-9a80-7a65f79b7df2 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:58.935896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0807f955-a02f-43b6-b18a-5218827640d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:58.936582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a6fae-52d8-4c44-b7e3-e4ed9ae7ebbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:58.937194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a6fae-52d8-4c44-b7e3-e4ed9ae7ebbd None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:58.937469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a6fae-52d8-4c44-b7e3-e4ed9ae7ebbd None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:58.937737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a6fae-52d8-4c44-b7e3-e4ed9ae7ebbd None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:58.938388 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a6fae-52d8-4c44-b7e3-e4ed9ae7ebbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:58.938686 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a6fae-52d8-4c44-b7e3-e4ed9ae7ebbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:58.939829 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807f955-a02f-43b6-b18a-5218827640d1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:58.940098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807f955-a02f-43b6-b18a-5218827640d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:58.940364 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807f955-a02f-43b6-b18a-5218827640d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:58.941880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a6fae-52d8-4c44-b7e3-e4ed9ae7ebbd None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:58.955449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807f955-a02f-43b6-b18a-5218827640d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:58.957277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0807f955-a02f-43b6-b18a-5218827640d1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:58.959330 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 228/463] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:58.960360 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a6fae-52d8-4c44-b7e3-e4ed9ae7ebbd None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:58.972012 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:58.974610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0807f955-a02f-43b6-b18a-5218827640d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:53:58.974610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:58.974610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:58.974610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:58.974610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:58.974610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:58.974610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:58.974610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:58.974610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:58.974610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:58.974610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:58.974610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:58.974610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:53:58.974610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:53:58.974610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:53:58.974610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:58.975565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:53:58.977166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:53:58.977166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:53:58.977166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:53:58.977166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 236/464] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:53:58.978004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10108ff8-6560-4112-813d-f97b57bc40a8 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:58.978263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10108ff8-6560-4112-813d-f97b57bc40a8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:58.978535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10108ff8-6560-4112-813d-f97b57bc40a8 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:58.979907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 237/465] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:58.983206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10108ff8-6560-4112-813d-f97b57bc40a8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:59.072956 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a6fae-52d8-4c44-b7e3-e4ed9ae7ebbd None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:59.103407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10108ff8-6560-4112-813d-f97b57bc40a8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:59.109140 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a5030-b77e-4c6e-92b4-5999c588ee51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:59.109771 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a5030-b77e-4c6e-92b4-5999c588ee51 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:59.110046 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a5030-b77e-4c6e-92b4-5999c588ee51 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:59.110314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a5030-b77e-4c6e-92b4-5999c588ee51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:59.121097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a5030-b77e-4c6e-92b4-5999c588ee51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:59.122548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a5030-b77e-4c6e-92b4-5999c588ee51 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:59.140337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-de3a5030-b77e-4c6e-92b4-5999c588ee51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:53:59.140337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:53:59.140337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:53:59.140337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:53:59.140337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:53:59.140337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:53:59.140337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:53:59.140337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:53:59.140337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:53:59.140337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:53:59.140337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:53:59.140337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:53:59.140337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:53:59.140337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:53:59.140337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:53:59.140337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:53:59.141370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:53:59.142797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:53:59.142797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:53:59.142797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:53:59.144247 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 229/466] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:53:59.144924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09a2bc-b88f-4ad9-a2e9-0776b4259791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:59.145362 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a5030-b77e-4c6e-92b4-5999c588ee51 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:59.145553 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09a2bc-b88f-4ad9-a2e9-0776b4259791 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:59.145834 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09a2bc-b88f-4ad9-a2e9-0776b4259791 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:59.146103 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09a2bc-b88f-4ad9-a2e9-0776b4259791 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:59.146764 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e09a2bc-b88f-4ad9-a2e9-0776b4259791 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:59.148833 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e09a2bc-b88f-4ad9-a2e9-0776b4259791 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:59.220780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:59.263370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a5030-b77e-4c6e-92b4-5999c588ee51 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:59.263709 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 238/467] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:53:58 2022] POST /identity/v3/users => generated 246 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:53:59.265118 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0814ba-ad0f-4f5b-80f8-83fc7a232d8b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:59.265454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0814ba-ad0f-4f5b-80f8-83fc7a232d8b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:59.265728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0814ba-ad0f-4f5b-80f8-83fc7a232d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:59.285929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bc0814ba-ad0f-4f5b-80f8-83fc7a232d8b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:59.287201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bc0814ba-ad0f-4f5b-80f8-83fc7a232d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:59.298911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:59.329219 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93685db9-388c-4d09-94a8-acc96b80ff3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:59.329750 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93685db9-388c-4d09-94a8-acc96b80ff3f None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:59.330029 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93685db9-388c-4d09-94a8-acc96b80ff3f None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:59.330304 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93685db9-388c-4d09-94a8-acc96b80ff3f None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:59.330997 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93685db9-388c-4d09-94a8-acc96b80ff3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:59.331307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93685db9-388c-4d09-94a8-acc96b80ff3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:59.332779 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93685db9-388c-4d09-94a8-acc96b80ff3f None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:59.348796 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 230/468] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:59.349818 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93685db9-388c-4d09-94a8-acc96b80ff3f None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:59.391417 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 239/469] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:59.393499 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0814ba-ad0f-4f5b-80f8-83fc7a232d8b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:59.505700 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93685db9-388c-4d09-94a8-acc96b80ff3f None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:59.549950 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0814ba-ad0f-4f5b-80f8-83fc7a232d8b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:59.551333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:59.572222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ad248-6ede-483f-a1f5-265e8d5afe0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:59.572833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ad248-6ede-483f-a1f5-265e8d5afe0c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:59.573104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ad248-6ede-483f-a1f5-265e8d5afe0c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:59.573367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ad248-6ede-483f-a1f5-265e8d5afe0c None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:59.574017 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ad248-6ede-483f-a1f5-265e8d5afe0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:53:59.574332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ad248-6ede-483f-a1f5-265e8d5afe0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:59.576483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ad248-6ede-483f-a1f5-265e8d5afe0c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:59.589119 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 231/470] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:53:59.599087 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ad248-6ede-483f-a1f5-265e8d5afe0c None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:59.600069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f65ff3-b6a6-4cd4-bc04-3a2476d80e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:59.600684 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f65ff3-b6a6-4cd4-bc04-3a2476d80e0c None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:59.601482 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f65ff3-b6a6-4cd4-bc04-3a2476d80e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:59.601807 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f65ff3-b6a6-4cd4-bc04-3a2476d80e0c None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:59.602400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f65ff3-b6a6-4cd4-bc04-3a2476d80e0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:59.603887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f65ff3-b6a6-4cd4-bc04-3a2476d80e0c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:59.682139 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:53:59.689408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 240/471] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:53:59 2022] POST /identity/v3/users => generated 248 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:53:59.690657 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79061bf4-28f4-421f-9023-4a6e7c775387 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:59.690951 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79061bf4-28f4-421f-9023-4a6e7c775387 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:59.691230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79061bf4-28f4-421f-9023-4a6e7c775387 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:59.693510 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 241/472] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:53:59.699357 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad3a22-8ee0-4f52-ae70-6434fe38b022 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:59.699642 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad3a22-8ee0-4f52-ae70-6434fe38b022 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:59.699909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad3a22-8ee0-4f52-ae70-6434fe38b022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:59.735913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f9ad3a22-8ee0-4f52-ae70-6434fe38b022 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:53:59.737423 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f9ad3a22-8ee0-4f52-ae70-6434fe38b022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:53:59.743633 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ad248-6ede-483f-a1f5-265e8d5afe0c None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:59.792152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cafaf8-bf76-496e-8fde-4b9e6eacb855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:53:59.792808 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cafaf8-bf76-496e-8fde-4b9e6eacb855 None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:53:59.793080 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cafaf8-bf76-496e-8fde-4b9e6eacb855 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:53:59.793349 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cafaf8-bf76-496e-8fde-4b9e6eacb855 None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:53:59.794029 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cafaf8-bf76-496e-8fde-4b9e6eacb855 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:53:59.795527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cafaf8-bf76-496e-8fde-4b9e6eacb855 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:53:59.834552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 242/473] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:53:59.836296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad3a22-8ee0-4f52-ae70-6434fe38b022 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:53:59.925545 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 232/474] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:53:59 2022] POST /identity/v3/users => generated 248 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 07:53:59.967612 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad3a22-8ee0-4f52-ae70-6434fe38b022 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:53:59.997422 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:00.006250 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103dba3-3ee9-41f1-868c-4a636b8ab196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:00.007374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103dba3-3ee9-41f1-868c-4a636b8ab196 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:00.007820 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103dba3-3ee9-41f1-868c-4a636b8ab196 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:00.008334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103dba3-3ee9-41f1-868c-4a636b8ab196 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:00.020536 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103dba3-3ee9-41f1-868c-4a636b8ab196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:00.022525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103dba3-3ee9-41f1-868c-4a636b8ab196 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:00.041189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-8103dba3-3ee9-41f1-868c-4a636b8ab196 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:00.041189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:00.041189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:00.041189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:00.041189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:00.041189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:00.041189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:00.041189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:00.041189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:00.041189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:00.041189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:00.041189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:00.041189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:00.041189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:00.041189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:00.041189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:00.043742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:00.044623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:00.044623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:00.044623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:00.050249 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 243/475] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:00.052019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cafaf8-bf76-496e-8fde-4b9e6eacb855 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:00.093361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:00.135571 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103dba3-3ee9-41f1-868c-4a636b8ab196 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:00.190146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cafaf8-bf76-496e-8fde-4b9e6eacb855 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:00.231532 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856823e-4783-4e2a-8383-e5ee398d4026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:00.232774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856823e-4783-4e2a-8383-e5ee398d4026 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:00.233310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856823e-4783-4e2a-8383-e5ee398d4026 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:00.234396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856823e-4783-4e2a-8383-e5ee398d4026 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:00.235566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856823e-4783-4e2a-8383-e5ee398d4026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:00.236074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856823e-4783-4e2a-8383-e5ee398d4026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:00.238833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856823e-4783-4e2a-8383-e5ee398d4026 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:00.258967 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 233/476] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:00.262037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856823e-4783-4e2a-8383-e5ee398d4026 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:00.289019 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103dba3-3ee9-41f1-868c-4a636b8ab196 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:00.338622 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cb380-d839-4678-8d67-5f03b2c4e8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:00.339687 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cb380-d839-4678-8d67-5f03b2c4e8e7 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:00.339965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cb380-d839-4678-8d67-5f03b2c4e8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:00.340234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cb380-d839-4678-8d67-5f03b2c4e8e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:00.340918 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718cb380-d839-4678-8d67-5f03b2c4e8e7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:00.342285 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718cb380-d839-4678-8d67-5f03b2c4e8e7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:00.382921 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:00.389195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856823e-4783-4e2a-8383-e5ee398d4026 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:00.394822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.sql.core [None req-718cb380-d839-4678-8d67-5f03b2c4e8e7 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 06 07:54:00.394822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 06 07:54:00.394822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [parameters: {'id': '0e4db21b83bc40b5a4cfaf7f07c46c6c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 06 07:54:00.394822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81480) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 06 07:54:00.411649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-718cb380-d839-4678-8d67-5f03b2c4e8e7 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 06 07:54:00.411649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:00.411649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 07:54:00.411649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 07:54:00.411649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 07:54:00.411649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 07:54:00.411649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 07:54:00.411649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 07:54:00.411649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 07:54:00.411649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application conn.query(q) Oct 06 07:54:00.411649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 07:54:00.411649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 07:54:00.411649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 07:54:00.411649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result.read() Oct 06 07:54:00.411649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return ref.to_dict() Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 07:54:00.418408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application next(self.gen) Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application yield resource Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application next(self.gen) Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.session.rollback() Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.force_reraise() Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise self.value Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application session.commit() Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 06 07:54:00.419465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._prepare_impl() Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.session.flush() Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._flush(objects) Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application compat.raise_( Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application flush_context.execute() Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rec.execute(self) Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 06 07:54:00.420347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 06 07:54:00.421757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 06 07:54:00.421757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 06 07:54:00.421757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 06 07:54:00.421757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 06 07:54:00.421757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 06 07:54:00.421757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 06 07:54:00.421757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 06 07:54:00.421757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 06 07:54:00.421757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 06 07:54:00.421757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 06 07:54:00.421757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 06 07:54:00.421757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 06 07:54:00.421757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 06 07:54:00.421757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 06 07:54:00.421757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 07:54:00.421757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception Oct 06 07:54:00.423728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 07:54:00.423728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 07:54:00.423728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 07:54:00.423728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 07:54:00.423728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 07:54:00.423728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 07:54:00.423728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 07:54:00.423728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application conn.query(q) Oct 06 07:54:00.423728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 07:54:00.423728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 07:54:00.423728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 07:54:00.423728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result.read() Oct 06 07:54:00.423728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 07:54:00.423728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 07:54:00.423728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 07:54:00.423728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 07:54:00.424650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 07:54:00.424650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 07:54:00.424650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 07:54:00.424650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 07:54:00.424650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 06 07:54:00.424650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 06 07:54:00.424650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [parameters: {'id': '0e4db21b83bc40b5a4cfaf7f07c46c6c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 06 07:54:00.424650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 06 07:54:00.424650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:00.424650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:00.424650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:00.424650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:00.424650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:00.424650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:00.424650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:00.424650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:00.425484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:00.425484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:00.425484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:00.425484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:00.425484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:00.425484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:00.425484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 06 07:54:00.425484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 06 07:54:00.425484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:00.425484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:00.425484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 06 07:54:00.425484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 06 07:54:00.425484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 06 07:54:00.425484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 06 07:54:00.425484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 06 07:54:00.425484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:00.426336 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 244/477] 10.222.0.30 () {60 vars in 1131 bytes} [Thu Oct 6 07:54:00 2022] POST /identity/v3/roles => generated 144 bytes in 278 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:00.426336 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cb380-d839-4678-8d67-5f03b2c4e8e7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:00.462971 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670846b8-0d1e-44e6-945f-afee1c2c3d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:00.463821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670846b8-0d1e-44e6-945f-afee1c2c3d6a None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:00.464247 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670846b8-0d1e-44e6-945f-afee1c2c3d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:00.464689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670846b8-0d1e-44e6-945f-afee1c2c3d6a None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:00.465490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670846b8-0d1e-44e6-945f-afee1c2c3d6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:00.468561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670846b8-0d1e-44e6-945f-afee1c2c3d6a None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:00.492413 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:00.550278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cb380-d839-4678-8d67-5f03b2c4e8e7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:00.583549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e14748-e9a8-4b61-b423-95c7fb0bab74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:00.584177 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e14748-e9a8-4b61-b423-95c7fb0bab74 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:00.584447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e14748-e9a8-4b61-b423-95c7fb0bab74 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:00.584708 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e14748-e9a8-4b61-b423-95c7fb0bab74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:00.596325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e14748-e9a8-4b61-b423-95c7fb0bab74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:00.602742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-81e14748-e9a8-4b61-b423-95c7fb0bab74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:54:00.602742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:00.602742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:00.602742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:00.602742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:00.602742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:00.602742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:00.602742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:00.602742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:00.602742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:00.602742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:00.602742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:00.602742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:00.602742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:00.602742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:00.602742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:00.603729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:00.603729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:00.603729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:00.603729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:00.603729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:00.603729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:00.603729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:00.603729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:00.603729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:00.603729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:00.603729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:00.603729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:00.603729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:00.603729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:00.603729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:00.603729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:00.604729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:00.604729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:00.604729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:00.604729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:00.604729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:00.604729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:00.604729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:00.604729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:00.604729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:00.604729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:00.604729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 06 07:54:00.604729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:00.604729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 245/478] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:54:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:00.612108 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e14748-e9a8-4b61-b423-95c7fb0bab74 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:00.629469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 234/479] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:54:00 2022] POST /identity/v3/users => generated 249 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:54:00.752636 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e14748-e9a8-4b61-b423-95c7fb0bab74 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:00.756952 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:00.805306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2c36d-2c55-4435-9550-b1a133648f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:00.805941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2c36d-2c55-4435-9550-b1a133648f45 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:00.806218 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2c36d-2c55-4435-9550-b1a133648f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:00.806561 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2c36d-2c55-4435-9550-b1a133648f45 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:00.807295 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2c36d-2c55-4435-9550-b1a133648f45 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:00.807596 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2c36d-2c55-4435-9550-b1a133648f45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:00.809023 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2c36d-2c55-4435-9550-b1a133648f45 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:00.824207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 246/480] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:01.999477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d0624-70ba-48cc-bb6a-5ed4479cd341 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:01.999986 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d0624-70ba-48cc-bb6a-5ed4479cd341 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:02.000085 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d0624-70ba-48cc-bb6a-5ed4479cd341 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:02.001652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 235/481] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:02.006791 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c3c2a-3a4d-40c0-8aa2-6e03181be0e1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:02.007314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c3c2a-3a4d-40c0-8aa2-6e03181be0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:02.007713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c3c2a-3a4d-40c0-8aa2-6e03181be0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:02.032688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-370c3c2a-3a4d-40c0-8aa2-6e03181be0e1 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:02.034210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-370c3c2a-3a4d-40c0-8aa2-6e03181be0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:02.124518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 247/482] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:02.539502 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d0624-70ba-48cc-bb6a-5ed4479cd341 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:02.638575 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac705c-3453-41fb-a7ed-1489b1dd8e3c None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:02.639221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac705c-3453-41fb-a7ed-1489b1dd8e3c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:02.639689 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac705c-3453-41fb-a7ed-1489b1dd8e3c None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:02.641569 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 248/483] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:54:02.646593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77b60c-a08c-490a-ab4c-5505be5daf9d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:02.647130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77b60c-a08c-490a-ab4c-5505be5daf9d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:02.647601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77b60c-a08c-490a-ab4c-5505be5daf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:02.666390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dc77b60c-a08c-490a-ab4c-5505be5daf9d None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:02.667923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dc77b60c-a08c-490a-ab4c-5505be5daf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:02.678740 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d0624-70ba-48cc-bb6a-5ed4479cd341 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:02.726806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2b72e-3609-4f81-aff2-568596d50be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:02.727172 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2b72e-3609-4f81-aff2-568596d50be2 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:02.727375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2b72e-3609-4f81-aff2-568596d50be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:02.727645 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2b72e-3609-4f81-aff2-568596d50be2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:02.738147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2b72e-3609-4f81-aff2-568596d50be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:02.744773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-f7b2b72e-3609-4f81-aff2-568596d50be2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:02.744773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:02.744773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:02.744773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:02.744773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:02.744773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:02.744773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:02.744773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:02.744773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:02.744773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:02.744773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:02.744773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:02.744773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:02.744773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:02.744773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:02.744773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:02.745746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:02.745746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:02.745746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:02.745746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:02.745746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:02.745746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:02.745746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:02.745746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:02.745746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:02.745746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:02.745746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:02.745746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:02.745746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:02.745746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:02.745746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:02.745746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:02.746554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:02.746554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:02.746554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:02.746554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:02.746554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:02.746554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:02.746554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:02.746554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:02.746554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:02.746554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:02.746554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:02.746554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:02.746554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 236/484] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:02.747813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75c9cb-3ada-4aa0-808c-79437332adb5 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:02.747919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75c9cb-3ada-4aa0-808c-79437332adb5 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:02.748103 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75c9cb-3ada-4aa0-808c-79437332adb5 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:02.749806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 237/485] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:02.751188 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 249/486] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:54:02.758988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75c9cb-3ada-4aa0-808c-79437332adb5 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:02.764554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b0d38-06a2-4a4c-9a0d-80ba1063ef7c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:02.764836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b0d38-06a2-4a4c-9a0d-80ba1063ef7c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:02.765109 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b0d38-06a2-4a4c-9a0d-80ba1063ef7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:02.783308 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:02.800352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-019b0d38-06a2-4a4c-9a0d-80ba1063ef7c None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:02.800352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-019b0d38-06a2-4a4c-9a0d-80ba1063ef7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:02.825829 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:02.886129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 250/487] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:02.956165 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75c9cb-3ada-4aa0-808c-79437332adb5 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:02.997883 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14431622-31a2-4229-b502-71e461990f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:02.998741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14431622-31a2-4229-b502-71e461990f23 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:02.999039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14431622-31a2-4229-b502-71e461990f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:02.999309 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14431622-31a2-4229-b502-71e461990f23 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:03.000033 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14431622-31a2-4229-b502-71e461990f23 None admin] RBAC: Adding query filter params (name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:03.000549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14431622-31a2-4229-b502-71e461990f23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:03.001751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14431622-31a2-4229-b502-71e461990f23 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:03.015286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 238/488] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:03.020635 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b0d38-06a2-4a4c-9a0d-80ba1063ef7c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:03.072302 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14431622-31a2-4229-b502-71e461990f23 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:03.153971 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b0d38-06a2-4a4c-9a0d-80ba1063ef7c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:03.193020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356563d9-154e-4db6-92b8-ab5c408c3659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:03.193645 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356563d9-154e-4db6-92b8-ab5c408c3659 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:03.193917 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356563d9-154e-4db6-92b8-ab5c408c3659 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:03.194183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356563d9-154e-4db6-92b8-ab5c408c3659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:03.205844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356563d9-154e-4db6-92b8-ab5c408c3659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:03.207346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356563d9-154e-4db6-92b8-ab5c408c3659 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:03.224120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-356563d9-154e-4db6-92b8-ab5c408c3659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:03.224120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:03.224120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:03.224120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:03.224120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:03.224120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:03.224120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:03.224120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:03.224120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:03.224120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:03.224120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:03.224120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:03.224120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:03.224120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:03.224120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:03.224120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:03.225028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:03.226510 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:03.226510 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:03.226510 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:03.226510 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 251/489] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:03.239901 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356563d9-154e-4db6-92b8-ab5c408c3659 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:03.242675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:03.252348 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:03.311795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14431622-31a2-4229-b502-71e461990f23 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:03.362476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ef674-afcf-4a87-b3c9-aabd3c52b979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:03.363485 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ef674-afcf-4a87-b3c9-aabd3c52b979 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:03.363872 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356563d9-154e-4db6-92b8-ab5c408c3659 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:03.364210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ef674-afcf-4a87-b3c9-aabd3c52b979 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:03.364677 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ef674-afcf-4a87-b3c9-aabd3c52b979 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:03.376608 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ef674-afcf-4a87-b3c9-aabd3c52b979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:03.383044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-507ef674-afcf-4a87-b3c9-aabd3c52b979 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:03.383044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:03.383044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:03.383044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:03.383044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:03.383044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:03.383044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:03.383044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:03.383044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:03.383044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:03.383044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:03.383044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:03.383044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:03.383044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:03.383044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:03.383044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:03.383926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:03.383926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:03.383926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:03.383926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:03.383926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:03.383926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:03.383926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:03.383926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:03.383926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:03.383926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:03.383926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:03.383926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:03.383926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:03.383926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:03.383926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:03.383926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:03.384739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:03.384739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:03.384739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:03.384739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:03.384739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:03.384739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:03.384739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:03.384739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:03.384739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:03.384739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:03.384739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:03.384739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:03.391361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 239/490] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:03.392542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ef674-afcf-4a87-b3c9-aabd3c52b979 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:03.401943 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9e03b-cee8-47ca-affe-07b11a7eb1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:03.402544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9e03b-cee8-47ca-affe-07b11a7eb1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:03.402816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9e03b-cee8-47ca-affe-07b11a7eb1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:03.403117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9e03b-cee8-47ca-affe-07b11a7eb1f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:03.403824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9e03b-cee8-47ca-affe-07b11a7eb1f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:03.404493 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9e03b-cee8-47ca-affe-07b11a7eb1f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:03.405725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9e03b-cee8-47ca-affe-07b11a7eb1f4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:03.419437 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 252/491] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:03.420124 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9e03b-cee8-47ca-affe-07b11a7eb1f4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:03.495517 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:03.505197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ef674-afcf-4a87-b3c9-aabd3c52b979 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:03.531885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9e03b-cee8-47ca-affe-07b11a7eb1f4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:03.547039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3b65a-cdfd-4451-a9e7-6aa5f984d9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:03.547664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3b65a-cdfd-4451-a9e7-6aa5f984d9e6 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:03.547946 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3b65a-cdfd-4451-a9e7-6aa5f984d9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:03.548217 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3b65a-cdfd-4451-a9e7-6aa5f984d9e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:03.551034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:03.560202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3b65a-cdfd-4451-a9e7-6aa5f984d9e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:03.565138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-f8d3b65a-cdfd-4451-a9e7-6aa5f984d9e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:03.565138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:03.565138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:03.565138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:03.565138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:03.565138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:03.565138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:03.565138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:03.565138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:03.565138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:03.565138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:03.565138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:03.565138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:03.565138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:03.565138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:03.565138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:03.566052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:03.566052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:03.566052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:03.566052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:03.566052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:03.566052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:03.566052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:03.566052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:03.566052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:03.566052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:03.566052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:03.566052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:03.566052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:03.566052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:03.566052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:03.566052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:03.566877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:03.566877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:03.566877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:03.566877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:03.566877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:03.566877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:03.566877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:03.566877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:03.566877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:03.566877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:03.566877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:03.566877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:03.566877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 240/492] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:03.568095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3b65a-cdfd-4451-a9e7-6aa5f984d9e6 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:03.574703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248cc78-2a92-4bdc-a153-2e0af2f60819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:03.575349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248cc78-2a92-4bdc-a153-2e0af2f60819 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:03.575623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248cc78-2a92-4bdc-a153-2e0af2f60819 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:03.575890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248cc78-2a92-4bdc-a153-2e0af2f60819 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:03.576558 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9248cc78-2a92-4bdc-a153-2e0af2f60819 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:03.576854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9248cc78-2a92-4bdc-a153-2e0af2f60819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:03.578219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9248cc78-2a92-4bdc-a153-2e0af2f60819 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:03.592221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 253/493] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:03.592221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cbfac-faf2-4017-b061-befc4b0909c3 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:03.592221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cbfac-faf2-4017-b061-befc4b0909c3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:03.592221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cbfac-faf2-4017-b061-befc4b0909c3 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:03.592221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 254/494] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:03.592221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68cbfac-faf2-4017-b061-befc4b0909c3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:03.677535 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3b65a-cdfd-4451-a9e7-6aa5f984d9e6 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:03.703801 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68cbfac-faf2-4017-b061-befc4b0909c3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:03.714879 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36859f-6091-4e8e-a665-b5c39d3ab039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:03.715535 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36859f-6091-4e8e-a665-b5c39d3ab039 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:03.715812 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36859f-6091-4e8e-a665-b5c39d3ab039 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:03.716083 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36859f-6091-4e8e-a665-b5c39d3ab039 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:03.724847 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36859f-6091-4e8e-a665-b5c39d3ab039 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:03.726511 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36859f-6091-4e8e-a665-b5c39d3ab039 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:03.737062 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:03.740316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-8a36859f-6091-4e8e-a665-b5c39d3ab039 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:54:03.740316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:03.740316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:03.740316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:03.740316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:03.740316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:03.740316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:03.740316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:03.740316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:03.740316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:03.740316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:03.740316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:03.740316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:03.740316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:03.740316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:03.740316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:03.741259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:03.743692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:03.743692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:03.743692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:03.743692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:03.743692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:03.743692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:03.743692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:03.743692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:03.743692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:54:03.743692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:03.743692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 241/495] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Oct 6 07:54:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:03.743692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36859f-6091-4e8e-a665-b5c39d3ab039 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:03.747098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd68ce9-23da-4b59-ba63-2d3da4d76d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:03.749641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd68ce9-23da-4b59-ba63-2d3da4d76d05 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:03.749641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd68ce9-23da-4b59-ba63-2d3da4d76d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:03.749641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd68ce9-23da-4b59-ba63-2d3da4d76d05 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:03.749641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd68ce9-23da-4b59-ba63-2d3da4d76d05 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:03.749641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd68ce9-23da-4b59-ba63-2d3da4d76d05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:03.751111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd68ce9-23da-4b59-ba63-2d3da4d76d05 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:03.760727 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 255/496] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:03.761719 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964edac2-e8e2-4554-b33a-80a0a12ae877 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:03.762005 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964edac2-e8e2-4554-b33a-80a0a12ae877 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:03.762280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964edac2-e8e2-4554-b33a-80a0a12ae877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:03.781491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-964edac2-e8e2-4554-b33a-80a0a12ae877 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:03.782681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-964edac2-e8e2-4554-b33a-80a0a12ae877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:03.819132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:03.853921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 256/497] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:03.854773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964edac2-e8e2-4554-b33a-80a0a12ae877 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:03.867286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36859f-6091-4e8e-a665-b5c39d3ab039 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:03.901456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f015381-8177-4031-a84d-92ab7a135ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:03.902076 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f015381-8177-4031-a84d-92ab7a135ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:03.902356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f015381-8177-4031-a84d-92ab7a135ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:03.902621 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f015381-8177-4031-a84d-92ab7a135ee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:03.913785 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f015381-8177-4031-a84d-92ab7a135ee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:03.915261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f015381-8177-4031-a84d-92ab7a135ee6 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:03.930111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-1f015381-8177-4031-a84d-92ab7a135ee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:03.930111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:03.930111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:03.930111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:03.930111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:03.930111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:03.930111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:03.930111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:03.930111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:03.930111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:03.930111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:03.930111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:03.930111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:03.930111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:03.930111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:03.930111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:03.931018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:03.931872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:03.931872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:03.931872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:03.931872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 242/498] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:03.935806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f015381-8177-4031-a84d-92ab7a135ee6 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:03.961543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964edac2-e8e2-4554-b33a-80a0a12ae877 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:03.988183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:04.001226 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2869bb-4a3d-4932-a9c1-d84ef022ac1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:04.001871 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2869bb-4a3d-4932-a9c1-d84ef022ac1d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:04.002140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2869bb-4a3d-4932-a9c1-d84ef022ac1d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:04.002403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2869bb-4a3d-4932-a9c1-d84ef022ac1d None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:04.003156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2869bb-4a3d-4932-a9c1-d84ef022ac1d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:04.003456 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2869bb-4a3d-4932-a9c1-d84ef022ac1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:04.004880 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2869bb-4a3d-4932-a9c1-d84ef022ac1d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:04.044219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 257/499] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Oct 6 07:54:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:04.044844 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f015381-8177-4031-a84d-92ab7a135ee6 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:04.046616 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5768d-c3b3-44e9-8735-feffef4178d1 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:04.047101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5768d-c3b3-44e9-8735-feffef4178d1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:04.047545 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5768d-c3b3-44e9-8735-feffef4178d1 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:04.049397 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 258/500] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:04.050086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5768d-c3b3-44e9-8735-feffef4178d1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:04.060394 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:04.091453 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6518c5f-a0be-40b6-a2b8-889666554706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:04.092078 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6518c5f-a0be-40b6-a2b8-889666554706 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:04.092356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6518c5f-a0be-40b6-a2b8-889666554706 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:04.092624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6518c5f-a0be-40b6-a2b8-889666554706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:04.106143 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6518c5f-a0be-40b6-a2b8-889666554706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:04.107979 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6518c5f-a0be-40b6-a2b8-889666554706 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:04.128595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-b6518c5f-a0be-40b6-a2b8-889666554706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:04.128595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:04.128595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:04.128595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:04.128595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:04.128595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:04.128595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:04.128595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:04.128595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:04.128595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:04.128595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:04.128595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:04.128595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:04.128595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:04.128595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:04.128595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:04.129547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:04.135974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:04.135974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:04.135974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:04.135974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 243/501] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:04.135974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6518c5f-a0be-40b6-a2b8-889666554706 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:04.161427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5768d-c3b3-44e9-8735-feffef4178d1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:04.195257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c295f6-0daa-481e-abb4-853052054974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:04.195778 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c295f6-0daa-481e-abb4-853052054974 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:04.196049 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c295f6-0daa-481e-abb4-853052054974 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:04.196313 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c295f6-0daa-481e-abb4-853052054974 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:04.196948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c295f6-0daa-481e-abb4-853052054974 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:04.197242 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c295f6-0daa-481e-abb4-853052054974 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:04.198700 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c295f6-0daa-481e-abb4-853052054974 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:04.209536 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 259/502] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:04.211400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab36d8-4852-4db1-8e59-3bf76f93021f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:04.211677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab36d8-4852-4db1-8e59-3bf76f93021f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:04.211942 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab36d8-4852-4db1-8e59-3bf76f93021f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:04.229467 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-72ab36d8-4852-4db1-8e59-3bf76f93021f None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:04.230666 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-72ab36d8-4852-4db1-8e59-3bf76f93021f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:04.235802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6518c5f-a0be-40b6-a2b8-889666554706 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:04.264637 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:04.270039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2f844-7e6b-413e-ba2e-31035528f087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:04.270639 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2f844-7e6b-413e-ba2e-31035528f087 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:04.270908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2f844-7e6b-413e-ba2e-31035528f087 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:04.271205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2f844-7e6b-413e-ba2e-31035528f087 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:04.297736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 260/503] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:04.297887 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:04.298824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab36d8-4852-4db1-8e59-3bf76f93021f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:04.345237 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2f844-7e6b-413e-ba2e-31035528f087 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:04.346720 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2f844-7e6b-413e-ba2e-31035528f087 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:04.408268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab36d8-4852-4db1-8e59-3bf76f93021f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:04.448900 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c5cd2-cde4-4613-a176-0dd8359ae8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:04.453982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 244/504] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:54:04.454138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c5cd2-cde4-4613-a176-0dd8359ae8fb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:04.454138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c5cd2-cde4-4613-a176-0dd8359ae8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:04.454138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c5cd2-cde4-4613-a176-0dd8359ae8fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:04.454370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2f844-7e6b-413e-ba2e-31035528f087 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:04.467006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c5cd2-cde4-4613-a176-0dd8359ae8fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:04.473259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c32c5cd2-cde4-4613-a176-0dd8359ae8fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:04.473259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:04.473259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:04.473259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:04.473259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:04.473259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:04.473259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:04.473259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:04.473259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:04.473259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:04.473259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:04.473259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:04.473259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:04.473259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:04.473259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:04.473259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:04.474136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:04.474136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:04.474136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:04.474136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:04.474136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:04.474136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:04.474136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:04.474136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:04.474136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:04.474136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:04.474136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:04.474136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:04.474136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:04.474136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:04.474136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:04.474136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:04.475084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:04.475084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:04.475084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:04.475084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:04.475084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:04.475084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:04.475084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:04.475084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:04.475084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:04.475084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:04.475084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:04.475084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:04.475084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 261/505] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:04.476953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c5cd2-cde4-4613-a176-0dd8359ae8fb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:04.509625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:04.513853 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:04.556926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2f844-7e6b-413e-ba2e-31035528f087 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:04.584597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c5cd2-cde4-4613-a176-0dd8359ae8fb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:04.593002 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23844feb-0c34-4dfc-b133-19b268383890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:04.593601 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23844feb-0c34-4dfc-b133-19b268383890 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:04.593881 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23844feb-0c34-4dfc-b133-19b268383890 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:04.594152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23844feb-0c34-4dfc-b133-19b268383890 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:04.594784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23844feb-0c34-4dfc-b133-19b268383890 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:04.595120 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23844feb-0c34-4dfc-b133-19b268383890 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:04.596512 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23844feb-0c34-4dfc-b133-19b268383890 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:04.607489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 245/506] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:04.608454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23844feb-0c34-4dfc-b133-19b268383890 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:04.624104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaede4a-ecc2-4f43-b239-fb69dffa92f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:04.624723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaede4a-ecc2-4f43-b239-fb69dffa92f6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:04.624999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaede4a-ecc2-4f43-b239-fb69dffa92f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:04.625263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaede4a-ecc2-4f43-b239-fb69dffa92f6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:04.632960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaede4a-ecc2-4f43-b239-fb69dffa92f6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:04.634380 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaede4a-ecc2-4f43-b239-fb69dffa92f6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:04.647474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1eaede4a-ecc2-4f43-b239-fb69dffa92f6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 07:54:04.647474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:04.647474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:04.647474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:04.647474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:04.647474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:04.647474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:04.647474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:04.647474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:04.647474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:04.647474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:04.647474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:04.647474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:04.647474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:04.647474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:04.647474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:04.648472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:04.649403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:04.649403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:04.649403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:04.649403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:04.649403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:04.649403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:04.649403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:04.649403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:04.649403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 07:54:04.649403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:04.649403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 262/507] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:54:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:04.649403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaede4a-ecc2-4f43-b239-fb69dffa92f6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:04.710978 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23844feb-0c34-4dfc-b133-19b268383890 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:04.727455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:04.741165 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:04.745692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66927a0-1c4c-4c9b-8ebc-2a7b5dc943b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:04.751318 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66927a0-1c4c-4c9b-8ebc-2a7b5dc943b4 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:04.751318 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66927a0-1c4c-4c9b-8ebc-2a7b5dc943b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:04.751318 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66927a0-1c4c-4c9b-8ebc-2a7b5dc943b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:04.751595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaede4a-ecc2-4f43-b239-fb69dffa92f6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:04.757174 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66927a0-1c4c-4c9b-8ebc-2a7b5dc943b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:04.758629 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66927a0-1c4c-4c9b-8ebc-2a7b5dc943b4 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:04.772596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-a66927a0-1c4c-4c9b-8ebc-2a7b5dc943b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:04.772596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:04.772596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:04.772596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:04.772596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:04.772596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:04.772596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:04.772596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:04.772596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:04.772596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:04.772596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:04.772596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:04.772596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:04.772596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:04.772596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:04.772596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:04.773708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:04.774542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:04.774542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:04.774542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:04.774542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 246/508] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:04.774894 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66927a0-1c4c-4c9b-8ebc-2a7b5dc943b4 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:04.789195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da30895-9755-49a5-b68e-74f02d4381da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:04.789792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da30895-9755-49a5-b68e-74f02d4381da None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:04.790074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da30895-9755-49a5-b68e-74f02d4381da None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:04.790501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da30895-9755-49a5-b68e-74f02d4381da None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:04.791188 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da30895-9755-49a5-b68e-74f02d4381da None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:04.791484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da30895-9755-49a5-b68e-74f02d4381da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:04.797257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da30895-9755-49a5-b68e-74f02d4381da None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:04.805189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 263/509] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:04.805189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da30895-9755-49a5-b68e-74f02d4381da None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:04.873585 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66927a0-1c4c-4c9b-8ebc-2a7b5dc943b4 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:04.900439 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da30895-9755-49a5-b68e-74f02d4381da None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:04.906275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f926d82-e9b0-4ecb-a70d-496133c0b531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:04.906889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f926d82-e9b0-4ecb-a70d-496133c0b531 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:04.907191 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f926d82-e9b0-4ecb-a70d-496133c0b531 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:04.907463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f926d82-e9b0-4ecb-a70d-496133c0b531 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:04.917570 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f926d82-e9b0-4ecb-a70d-496133c0b531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:04.919037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f926d82-e9b0-4ecb-a70d-496133c0b531 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:04.934156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-5f926d82-e9b0-4ecb-a70d-496133c0b531 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:04.934156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:04.934156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:04.934156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:04.934156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:04.934156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:04.934156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:04.934156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:04.934156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:04.934156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:04.934156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:04.934156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:04.934156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:04.934156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:04.934156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:04.934156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:04.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:04.936663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:04.936663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:04.936663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:04.936870 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34655f5e-2fb6-4af2-a4c8-5ff53be26cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:04.936870 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34655f5e-2fb6-4af2-a4c8-5ff53be26cbf None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:04.937044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 247/510] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:04.937044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f926d82-e9b0-4ecb-a70d-496133c0b531 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:04.939183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34655f5e-2fb6-4af2-a4c8-5ff53be26cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:04.939623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34655f5e-2fb6-4af2-a4c8-5ff53be26cbf None admin] PATH_INFO: `/v3/users/glance` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:04.948257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34655f5e-2fb6-4af2-a4c8-5ff53be26cbf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:04.949698 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34655f5e-2fb6-4af2-a4c8-5ff53be26cbf None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:04.962453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-34655f5e-2fb6-4af2-a4c8-5ff53be26cbf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 06 07:54:04.962453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:04.962453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:04.962453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:04.962453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:04.962453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:04.962453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:04.962453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:04.962453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:04.962453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:04.962453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:04.962453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:04.962453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:04.962453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:04.962453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:04.962453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:04.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:04.964298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:04.964298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:04.964298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:04.964298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:04.964298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:04.964298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:04.964298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:04.964298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:04.964298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 06 07:54:04.964298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:04.964824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:04.964894 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 264/511] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:54:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:04.964894 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34655f5e-2fb6-4af2-a4c8-5ff53be26cbf None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:04.982765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:05.037825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f926d82-e9b0-4ecb-a70d-496133c0b531 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:05.068050 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34655f5e-2fb6-4af2-a4c8-5ff53be26cbf None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:05.072709 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495b1d4-e3a1-46d2-80f7-057163fe5145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:05.073313 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495b1d4-e3a1-46d2-80f7-057163fe5145 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:05.073607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495b1d4-e3a1-46d2-80f7-057163fe5145 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:05.073866 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495b1d4-e3a1-46d2-80f7-057163fe5145 None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:05.074589 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495b1d4-e3a1-46d2-80f7-057163fe5145 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:05.074894 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495b1d4-e3a1-46d2-80f7-057163fe5145 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:05.076373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495b1d4-e3a1-46d2-80f7-057163fe5145 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:05.101617 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d0dad-22f3-4355-8ddd-79dcb546ccf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:05.102219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d0dad-22f3-4355-8ddd-79dcb546ccf2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:05.102493 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d0dad-22f3-4355-8ddd-79dcb546ccf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:05.102761 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d0dad-22f3-4355-8ddd-79dcb546ccf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:05.103435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d0dad-22f3-4355-8ddd-79dcb546ccf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:05.103736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d0dad-22f3-4355-8ddd-79dcb546ccf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:05.105098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d0dad-22f3-4355-8ddd-79dcb546ccf2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:05.111880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 248/512] 10.222.0.30 () {56 vars in 1137 bytes} [Thu Oct 6 07:54:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:05.112709 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495b1d4-e3a1-46d2-80f7-057163fe5145 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:05.120623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 265/513] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:05.121321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d0dad-22f3-4355-8ddd-79dcb546ccf2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:05.193692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:05.195082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:05.212890 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495b1d4-e3a1-46d2-80f7-057163fe5145 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:05.224427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d0dad-22f3-4355-8ddd-79dcb546ccf2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:05.248604 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5ab35-b581-4ccb-8b09-b1a536c5b215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:05.249238 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5ab35-b581-4ccb-8b09-b1a536c5b215 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:05.249514 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5ab35-b581-4ccb-8b09-b1a536c5b215 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:05.249786 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5ab35-b581-4ccb-8b09-b1a536c5b215 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:05.258702 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791dcf58-2df5-4629-9c9a-04c265eccf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:05.259324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dcf58-2df5-4629-9c9a-04c265eccf34 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:05.259598 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dcf58-2df5-4629-9c9a-04c265eccf34 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:05.259686 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5ab35-b581-4ccb-8b09-b1a536c5b215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:05.259863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791dcf58-2df5-4629-9c9a-04c265eccf34 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:05.260495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791dcf58-2df5-4629-9c9a-04c265eccf34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:05.260792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791dcf58-2df5-4629-9c9a-04c265eccf34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:05.261121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5ab35-b581-4ccb-8b09-b1a536c5b215 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:05.262251 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791dcf58-2df5-4629-9c9a-04c265eccf34 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:05.274247 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 266/514] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:05.275044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791dcf58-2df5-4629-9c9a-04c265eccf34 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:05.275870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-7de5ab35-b581-4ccb-8b09-b1a536c5b215 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:05.275870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:05.275870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:05.275870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:05.275870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:05.275870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:05.275870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:05.275870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:05.275870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:05.275870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:05.275870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:05.275870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:05.275870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:05.275870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:05.275870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:05.275870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:05.276737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:05.277593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:05.277593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:05.277593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:05.277593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 249/515] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:05.277816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5ab35-b581-4ccb-8b09-b1a536c5b215 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:05.376349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791dcf58-2df5-4629-9c9a-04c265eccf34 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:05.376659 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5ab35-b581-4ccb-8b09-b1a536c5b215 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:05.406161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:05.406491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:05.410992 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c2641-bffb-42e7-92b5-525c04df7c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:05.411491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11986239-54fb-4ed0-9634-43b229f3f48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:05.411606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c2641-bffb-42e7-92b5-525c04df7c19 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:05.411860 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c2641-bffb-42e7-92b5-525c04df7c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:05.412089 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11986239-54fb-4ed0-9634-43b229f3f48c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:05.412187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c2641-bffb-42e7-92b5-525c04df7c19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:05.412368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11986239-54fb-4ed0-9634-43b229f3f48c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:05.412638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11986239-54fb-4ed0-9634-43b229f3f48c None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:05.413387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11986239-54fb-4ed0-9634-43b229f3f48c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:05.413700 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11986239-54fb-4ed0-9634-43b229f3f48c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:05.415229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11986239-54fb-4ed0-9634-43b229f3f48c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:05.432240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 267/516] 10.222.0.30 () {56 vars in 1137 bytes} [Thu Oct 6 07:54:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:05.433046 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11986239-54fb-4ed0-9634-43b229f3f48c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:05.479420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c2641-bffb-42e7-92b5-525c04df7c19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:05.480795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c2641-bffb-42e7-92b5-525c04df7c19 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:05.531576 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11986239-54fb-4ed0-9634-43b229f3f48c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:05.564063 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545e00c-5689-4024-8c62-5e4df8c96aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:05.564687 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545e00c-5689-4024-8c62-5e4df8c96aaa None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:05.564966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545e00c-5689-4024-8c62-5e4df8c96aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:05.565241 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545e00c-5689-4024-8c62-5e4df8c96aaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:05.572381 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545e00c-5689-4024-8c62-5e4df8c96aaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:05.573811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545e00c-5689-4024-8c62-5e4df8c96aaa None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:05.576134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 250/517] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:54:05.576936 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c2641-bffb-42e7-92b5-525c04df7c19 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:05.590953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-9545e00c-5689-4024-8c62-5e4df8c96aaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:05.590953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:05.590953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:05.590953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:05.590953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:05.590953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:05.590953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:05.590953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:05.590953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:05.590953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:05.590953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:05.590953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:05.590953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:05.590953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:05.590953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:05.591869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:05.592715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:05.592715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:05.592715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:05.592715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:05.592715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:05.592715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:05.592715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:05.592715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:05.592715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 268/518] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:05.593181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545e00c-5689-4024-8c62-5e4df8c96aaa None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:05.610813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:05.641652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:05.678715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c2641-bffb-42e7-92b5-525c04df7c19 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:05.691608 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545e00c-5689-4024-8c62-5e4df8c96aaa None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:05.717375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461780c0-3c6a-49fd-ae35-ea2e5f1b4631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:05.717973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461780c0-3c6a-49fd-ae35-ea2e5f1b4631 None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:05.718251 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461780c0-3c6a-49fd-ae35-ea2e5f1b4631 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:05.718523 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461780c0-3c6a-49fd-ae35-ea2e5f1b4631 None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:05.719186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461780c0-3c6a-49fd-ae35-ea2e5f1b4631 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:05.720641 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461780c0-3c6a-49fd-ae35-ea2e5f1b4631 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:05.725606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea7c9d-99c6-4020-b9cd-c7b6f8169a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:05.726251 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea7c9d-99c6-4020-b9cd-c7b6f8169a9c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:05.726521 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea7c9d-99c6-4020-b9cd-c7b6f8169a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:05.726782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea7c9d-99c6-4020-b9cd-c7b6f8169a9c None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:05.727442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea7c9d-99c6-4020-b9cd-c7b6f8169a9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:05.727738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea7c9d-99c6-4020-b9cd-c7b6f8169a9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:05.729076 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea7c9d-99c6-4020-b9cd-c7b6f8169a9c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:05.740611 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 269/519] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:05.755998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea7c9d-99c6-4020-b9cd-c7b6f8169a9c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:05.811329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 251/520] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:54:05 2022] POST /identity/v3/users => generated 247 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:54:05.812394 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461780c0-3c6a-49fd-ae35-ea2e5f1b4631 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:05.852972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea7c9d-99c6-4020-b9cd-c7b6f8169a9c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:05.868779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:05.896161 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ba3cd-186c-4419-9ffd-81304c3ab388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:05.896783 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ba3cd-186c-4419-9ffd-81304c3ab388 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:05.897060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ba3cd-186c-4419-9ffd-81304c3ab388 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:05.897328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ba3cd-186c-4419-9ffd-81304c3ab388 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:05.916918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:05.931640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461780c0-3c6a-49fd-ae35-ea2e5f1b4631 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:05.972649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641f610-9143-4c54-b8d6-35fe01d009e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:05.973350 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641f610-9143-4c54-b8d6-35fe01d009e8 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:05.973614 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641f610-9143-4c54-b8d6-35fe01d009e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:05.973978 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641f610-9143-4c54-b8d6-35fe01d009e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:05.976420 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ba3cd-186c-4419-9ffd-81304c3ab388 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:05.978156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ba3cd-186c-4419-9ffd-81304c3ab388 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:05.986966 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641f610-9143-4c54-b8d6-35fe01d009e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:05.988451 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641f610-9143-4c54-b8d6-35fe01d009e8 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:06.006320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-0641f610-9143-4c54-b8d6-35fe01d009e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:06.006320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:06.006320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:06.006320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:06.006320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:06.006320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:06.006320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:06.006320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:06.006320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:06.006320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:06.006320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:06.006320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:06.006320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:06.006320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:06.006320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:06.006320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:06.008095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:06.009190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:06.009190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:06.009190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:06.009190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 252/521] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:06.009190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641f610-9143-4c54-b8d6-35fe01d009e8 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:06.097511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 270/522] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:54:06.098333 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ba3cd-186c-4419-9ffd-81304c3ab388 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:06.113218 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:06.131311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641f610-9143-4c54-b8d6-35fe01d009e8 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:06.174319 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103dcdf-eed9-45b2-bee1-5c3356a1f9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:06.174927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103dcdf-eed9-45b2-bee1-5c3356a1f9ac None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:06.175228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103dcdf-eed9-45b2-bee1-5c3356a1f9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:06.175496 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103dcdf-eed9-45b2-bee1-5c3356a1f9ac None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:06.176192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103dcdf-eed9-45b2-bee1-5c3356a1f9ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:06.176493 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103dcdf-eed9-45b2-bee1-5c3356a1f9ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:06.177921 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103dcdf-eed9-45b2-bee1-5c3356a1f9ac None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:06.192020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:06.196330 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 253/523] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:06.197028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103dcdf-eed9-45b2-bee1-5c3356a1f9ac None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:06.221027 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ba3cd-186c-4419-9ffd-81304c3ab388 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:06.260096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a72d1-9f1a-4d6e-994a-df361d5e53be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:06.260628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a72d1-9f1a-4d6e-994a-df361d5e53be None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:06.260902 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a72d1-9f1a-4d6e-994a-df361d5e53be None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:06.261167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a72d1-9f1a-4d6e-994a-df361d5e53be None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:06.268441 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a72d1-9f1a-4d6e-994a-df361d5e53be None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:06.269872 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a72d1-9f1a-4d6e-994a-df361d5e53be None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:06.282663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a71a72d1-9f1a-4d6e-994a-df361d5e53be None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 07:54:06.282663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:06.282663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:06.282663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:06.282663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:06.282663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:06.282663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:06.282663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:06.282663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:06.282663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:06.282663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:06.282663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:06.282663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:06.282663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:06.282663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:06.282663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:06.283725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:06.284624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:06.284624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:06.284624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:06.284624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:06.284624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:06.284624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:06.284624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:06.284624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:06.284624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 07:54:06.284624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:06.288442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 271/524] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:06.289276 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a72d1-9f1a-4d6e-994a-df361d5e53be None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:06.305632 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103dcdf-eed9-45b2-bee1-5c3356a1f9ac None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:06.344221 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159358a8-41d4-4f60-aeb4-c8435d3a6a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:06.344836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159358a8-41d4-4f60-aeb4-c8435d3a6a78 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:06.345116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159358a8-41d4-4f60-aeb4-c8435d3a6a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:06.345391 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159358a8-41d4-4f60-aeb4-c8435d3a6a78 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:06.346039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159358a8-41d4-4f60-aeb4-c8435d3a6a78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:06.346345 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159358a8-41d4-4f60-aeb4-c8435d3a6a78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:06.347766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159358a8-41d4-4f60-aeb4-c8435d3a6a78 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:06.360175 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 254/525] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:06.363814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:06.365283 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159358a8-41d4-4f60-aeb4-c8435d3a6a78 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:06.398356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a72d1-9f1a-4d6e-994a-df361d5e53be None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:06.431147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:06.434916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1b7af-79f9-4cc3-bf36-b348c7050a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:06.435862 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1b7af-79f9-4cc3-bf36-b348c7050a33 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:06.436145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1b7af-79f9-4cc3-bf36-b348c7050a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:06.436421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1b7af-79f9-4cc3-bf36-b348c7050a33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:06.449917 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1b7af-79f9-4cc3-bf36-b348c7050a33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:06.451428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1b7af-79f9-4cc3-bf36-b348c7050a33 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:06.466600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-88e1b7af-79f9-4cc3-bf36-b348c7050a33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:06.466600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:06.466600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:06.466600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:06.466600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:06.466600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:06.466600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:06.466600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:06.466600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:06.466600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:06.466600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:06.466600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:06.466600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:06.466600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:06.466600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:06.466600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:06.467670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:06.468540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:06.468540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:06.468540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:06.468540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 272/526] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:06.468540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1b7af-79f9-4cc3-bf36-b348c7050a33 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:06.475524 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159358a8-41d4-4f60-aeb4-c8435d3a6a78 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:06.512905 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30321d28-0987-41e4-af86-94e3dd74d60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:06.513424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30321d28-0987-41e4-af86-94e3dd74d60e None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:06.513693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30321d28-0987-41e4-af86-94e3dd74d60e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:06.514227 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30321d28-0987-41e4-af86-94e3dd74d60e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:06.514672 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30321d28-0987-41e4-af86-94e3dd74d60e None admin] RBAC: Adding query filter params (role.id=9f0e36c2ed8843fbabd3a4cf30bc3725, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=209795285384486183cda455fe316bc9) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:06.515023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30321d28-0987-41e4-af86-94e3dd74d60e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:06.516529 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30321d28-0987-41e4-af86-94e3dd74d60e None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:06.522351 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 255/527] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:06 2022] GET /identity/v3/role_assignments?user.id=209795285384486183cda455fe316bc9&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:06.531208 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30321d28-0987-41e4-af86-94e3dd74d60e None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:06.600694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1b7af-79f9-4cc3-bf36-b348c7050a33 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:06.630249 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:06.645890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed28ec-9d2f-4902-8d79-fde8590f3154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:06.646595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed28ec-9d2f-4902-8d79-fde8590f3154 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:06.646960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed28ec-9d2f-4902-8d79-fde8590f3154 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:06.647306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed28ec-9d2f-4902-8d79-fde8590f3154 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:06.648765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed28ec-9d2f-4902-8d79-fde8590f3154 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:06.649230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed28ec-9d2f-4902-8d79-fde8590f3154 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:06.650928 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed28ec-9d2f-4902-8d79-fde8590f3154 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:06.656314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30321d28-0987-41e4-af86-94e3dd74d60e None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:06.671479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 273/528] 10.222.0.30 () {56 vars in 1139 bytes} [Thu Oct 6 07:54:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:06.672354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed28ec-9d2f-4902-8d79-fde8590f3154 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:06.696567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:06.713977 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd9034-9d98-480f-bc0a-c15a2cde5984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:06.715675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd9034-9d98-480f-bc0a-c15a2cde5984 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:06.719276 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd9034-9d98-480f-bc0a-c15a2cde5984 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:06.719606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd9034-9d98-480f-bc0a-c15a2cde5984 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:06.732622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd9034-9d98-480f-bc0a-c15a2cde5984 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:06.734109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd9034-9d98-480f-bc0a-c15a2cde5984 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:06.747872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-26cd9034-9d98-480f-bc0a-c15a2cde5984 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:06.747872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:06.747872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:06.747872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:06.747872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:06.747872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:06.747872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:06.747872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:06.747872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:06.747872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:06.747872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:06.747872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:06.747872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:06.747872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:06.747872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:06.749084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:06.749931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:06.749931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:06.749931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:06.749931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:06.749931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:06.749931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:06.749931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:06.749931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:06.754293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 256/529] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:06.755363 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd9034-9d98-480f-bc0a-c15a2cde5984 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:06.810425 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed28ec-9d2f-4902-8d79-fde8590f3154 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:06.853463 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55473f09-df8c-411a-a8d3-30bd72ef7715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:06.854076 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55473f09-df8c-411a-a8d3-30bd72ef7715 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:06.854352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55473f09-df8c-411a-a8d3-30bd72ef7715 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:06.854621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55473f09-df8c-411a-a8d3-30bd72ef7715 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:06.855306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55473f09-df8c-411a-a8d3-30bd72ef7715 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:06.855604 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55473f09-df8c-411a-a8d3-30bd72ef7715 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:06.856967 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55473f09-df8c-411a-a8d3-30bd72ef7715 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:06.871933 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 274/530] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:06.873230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55473f09-df8c-411a-a8d3-30bd72ef7715 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:06.892579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd9034-9d98-480f-bc0a-c15a2cde5984 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:06.927403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:06.934793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d0700-1eb1-429e-87e7-acb973c3fc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:06.935427 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d0700-1eb1-429e-87e7-acb973c3fc08 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:06.935705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d0700-1eb1-429e-87e7-acb973c3fc08 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:06.935973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d0700-1eb1-429e-87e7-acb973c3fc08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:06.994682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:07.003212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55473f09-df8c-411a-a8d3-30bd72ef7715 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:07.022123 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d0700-1eb1-429e-87e7-acb973c3fc08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:07.025416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d0700-1eb1-429e-87e7-acb973c3fc08 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:07.041466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6a3a7-938a-4841-8374-c7502e45b620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:07.042103 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6a3a7-938a-4841-8374-c7502e45b620 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:07.042375 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6a3a7-938a-4841-8374-c7502e45b620 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:07.042640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6a3a7-938a-4841-8374-c7502e45b620 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:07.043482 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6a3a7-938a-4841-8374-c7502e45b620 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:07.043660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6a3a7-938a-4841-8374-c7502e45b620 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:07.045081 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6a3a7-938a-4841-8374-c7502e45b620 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:07.063573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 275/531] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:07.064604 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6a3a7-938a-4841-8374-c7502e45b620 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:07.167025 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 257/532] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:54:07.167886 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d0700-1eb1-429e-87e7-acb973c3fc08 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:07.188236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:07.192815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6a3a7-938a-4841-8374-c7502e45b620 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:07.235172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9a54d-429e-4273-aee2-81ca09c29148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:07.236447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9a54d-429e-4273-aee2-81ca09c29148 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:07.236955 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9a54d-429e-4273-aee2-81ca09c29148 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:07.237453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9a54d-429e-4273-aee2-81ca09c29148 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:07.247077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9a54d-429e-4273-aee2-81ca09c29148 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:07.249232 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9a54d-429e-4273-aee2-81ca09c29148 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:07.267737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-adb9a54d-429e-4273-aee2-81ca09c29148 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:07.267737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:07.267737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:07.267737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:07.267737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:07.267737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:07.267737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:07.267737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:07.267737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:07.267737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:07.267737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:07.267737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:07.267737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:07.267737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:07.267737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:07.268814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:07.269854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:07.269854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:07.269854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:07.269854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:07.269854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:07.269854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:07.269854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:07.269854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:07.273130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 276/533] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:07.273722 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:07.274394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9a54d-429e-4273-aee2-81ca09c29148 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:07.294158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d0700-1eb1-429e-87e7-acb973c3fc08 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:07.338971 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdb354-af7d-4837-827a-03795d56df81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:07.339849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdb354-af7d-4837-827a-03795d56df81 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:07.340432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdb354-af7d-4837-827a-03795d56df81 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:07.340876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdb354-af7d-4837-827a-03795d56df81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:07.342047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdb354-af7d-4837-827a-03795d56df81 None admin] RBAC: Adding query filter params (role.id=9f0e36c2ed8843fbabd3a4cf30bc3725, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=a899c6dabca948269f6c1370425f244b) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:07.342502 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdb354-af7d-4837-827a-03795d56df81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:07.344115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdb354-af7d-4837-827a-03795d56df81 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:07.352701 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 258/534] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:07 2022] GET /identity/v3/role_assignments?user.id=a899c6dabca948269f6c1370425f244b&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:07.353909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdb354-af7d-4837-827a-03795d56df81 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:07.399586 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9a54d-429e-4273-aee2-81ca09c29148 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:07.459120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9420e-da38-4bc9-9974-026142daaeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:07.463946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9420e-da38-4bc9-9974-026142daaeae None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:07.464442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9420e-da38-4bc9-9974-026142daaeae None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:07.464899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9420e-da38-4bc9-9974-026142daaeae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:07.487504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9420e-da38-4bc9-9974-026142daaeae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:07.490771 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9420e-da38-4bc9-9974-026142daaeae None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:07.533355 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdb354-af7d-4837-827a-03795d56df81 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:07.534182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a5c9420e-da38-4bc9-9974-026142daaeae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:07.534182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:07.534182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:07.534182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:07.534182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:07.534182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:07.534182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:07.534182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:07.534182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:07.534182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:07.534182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:07.534182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:07.534182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:07.534182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:07.534182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:07.534182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:07.535407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:07.536478 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:07.536478 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:07.536478 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:07.540476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 277/535] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:07.545976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9420e-da38-4bc9-9974-026142daaeae None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:07.577420 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:07.585691 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f70ac-a035-4361-9fcc-fe1b5fa3a5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:07.586588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f70ac-a035-4361-9fcc-fe1b5fa3a5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:07.587306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f70ac-a035-4361-9fcc-fe1b5fa3a5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:07.587766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f70ac-a035-4361-9fcc-fe1b5fa3a5a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:07.588775 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f70ac-a035-4361-9fcc-fe1b5fa3a5a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:07.592908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f70ac-a035-4361-9fcc-fe1b5fa3a5a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:07.592908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f70ac-a035-4361-9fcc-fe1b5fa3a5a0 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:07.607962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 259/536] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:07.613141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f70ac-a035-4361-9fcc-fe1b5fa3a5a0 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:07.631291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:07.727316 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9420e-da38-4bc9-9974-026142daaeae None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:07.738725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f70ac-a035-4361-9fcc-fe1b5fa3a5a0 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:07.779633 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed31ad8-35e4-4857-802a-71fc1c1d1492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:07.781011 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed31ad8-35e4-4857-802a-71fc1c1d1492 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:07.781441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed31ad8-35e4-4857-802a-71fc1c1d1492 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:07.781886 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed31ad8-35e4-4857-802a-71fc1c1d1492 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:07.782763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed31ad8-35e4-4857-802a-71fc1c1d1492 None admin] RBAC: Adding query filter params (role.id=9f0e36c2ed8843fbabd3a4cf30bc3725, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=30909d152a5c4a1fac6c1a2765868177) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:07.783240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed31ad8-35e4-4857-802a-71fc1c1d1492 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:07.784994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed31ad8-35e4-4857-802a-71fc1c1d1492 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:07.792563 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 260/537] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:07 2022] GET /identity/v3/role_assignments?user.id=30909d152a5c4a1fac6c1a2765868177&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:07.802866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a942a-f40a-4289-935a-2b20ce7f1fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:07.804359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a942a-f40a-4289-935a-2b20ce7f1fca None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:07.804844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a942a-f40a-4289-935a-2b20ce7f1fca None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:07.805264 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a942a-f40a-4289-935a-2b20ce7f1fca None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:07.806067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a942a-f40a-4289-935a-2b20ce7f1fca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:07.806540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a942a-f40a-4289-935a-2b20ce7f1fca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:07.808303 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a942a-f40a-4289-935a-2b20ce7f1fca None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:07.822554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 278/538] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:07.827812 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed31ad8-35e4-4857-802a-71fc1c1d1492 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:07.923845 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:07.956838 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed31ad8-35e4-4857-802a-71fc1c1d1492 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:07.998715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f9608-d754-4a58-866a-ac2b77035be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:07.999655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f9608-d754-4a58-866a-ac2b77035be4 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:08.000083 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f9608-d754-4a58-866a-ac2b77035be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:08.000524 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f9608-d754-4a58-866a-ac2b77035be4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:08.008019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f9608-d754-4a58-866a-ac2b77035be4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:08.009685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f9608-d754-4a58-866a-ac2b77035be4 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:08.027167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-6f3f9608-d754-4a58-866a-ac2b77035be4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:08.027167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:08.027167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:08.027167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:08.027167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:08.027167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:08.027167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:08.027167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:08.027167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:08.027167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:08.027167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:08.027167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:08.027167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:08.027167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:08.027167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:08.028192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:08.029245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:08.029245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:08.029245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:08.029245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:08.029245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:08.029245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:08.029245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:08.029245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:08.031418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 261/539] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:08.040221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a942a-f40a-4289-935a-2b20ce7f1fca None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:08.125517 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:08.163094 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a942a-f40a-4289-935a-2b20ce7f1fca None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:08.200422 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2db5e-d8bf-404b-aebc-6e44e7f4de44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:08.201032 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2db5e-d8bf-404b-aebc-6e44e7f4de44 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:08.201240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2db5e-d8bf-404b-aebc-6e44e7f4de44 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:08.201476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2db5e-d8bf-404b-aebc-6e44e7f4de44 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:08.202245 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2db5e-d8bf-404b-aebc-6e44e7f4de44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:08.202551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2db5e-d8bf-404b-aebc-6e44e7f4de44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:08.204054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2db5e-d8bf-404b-aebc-6e44e7f4de44 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:08.217175 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 279/540] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:08.222589 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f9608-d754-4a58-866a-ac2b77035be4 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:08.269627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc35341-0740-4bdc-b799-948f04f3a7d2 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:08.269941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc35341-0740-4bdc-b799-948f04f3a7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:08.270177 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc35341-0740-4bdc-b799-948f04f3a7d2 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:08.271814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 280/541] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:54:08.280344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c3cb2-a5eb-4d48-b1db-e8f99f6eb01a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:08.280628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c3cb2-a5eb-4d48-b1db-e8f99f6eb01a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:08.281189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c3cb2-a5eb-4d48-b1db-e8f99f6eb01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:08.303643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-777c3cb2-a5eb-4d48-b1db-e8f99f6eb01a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:08.305657 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-777c3cb2-a5eb-4d48-b1db-e8f99f6eb01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:08.389192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 281/542] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:08.418842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f9608-d754-4a58-866a-ac2b77035be4 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:08.480164 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:08.488181 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00fa95-02fc-4f2a-a701-35452b25fa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:08.488792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00fa95-02fc-4f2a-a701-35452b25fa78 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:08.489065 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00fa95-02fc-4f2a-a701-35452b25fa78 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:08.489329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00fa95-02fc-4f2a-a701-35452b25fa78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:08.491769 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00fa95-02fc-4f2a-a701-35452b25fa78 None admin] RBAC: Adding query filter params (role.id=9f0e36c2ed8843fbabd3a4cf30bc3725, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=bf6798f89acf44e897c481924edd9047) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:08.491769 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00fa95-02fc-4f2a-a701-35452b25fa78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:08.492055 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00fa95-02fc-4f2a-a701-35452b25fa78 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:08.500579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 262/543] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:08 2022] GET /identity/v3/role_assignments?user.id=bf6798f89acf44e897c481924edd9047&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:08.676053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c3cb2-a5eb-4d48-b1db-e8f99f6eb01a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:08.742780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:08.789002 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c3cb2-a5eb-4d48-b1db-e8f99f6eb01a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:08.825704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c9f56-1d59-4640-81d3-fa1a81b28032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:08.826410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c9f56-1d59-4640-81d3-fa1a81b28032 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:08.826689 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c9f56-1d59-4640-81d3-fa1a81b28032 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:08.826981 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c9f56-1d59-4640-81d3-fa1a81b28032 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:08.838770 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c9f56-1d59-4640-81d3-fa1a81b28032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:08.844277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c70c9f56-1d59-4640-81d3-fa1a81b28032 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:08.844277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:08.844277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:08.844277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:08.844277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:08.844277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:08.844277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:08.844277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:08.844277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:08.844277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:08.844277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:08.844277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:08.844277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:08.844277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:08.844277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:08.844277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:08.845199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:08.845199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:08.845199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:08.845199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:08.845199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:08.845199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:08.845199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:08.845199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:08.845199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:08.845199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:08.845199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:08.845199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:08.845199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:08.845199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:08.845199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:08.845199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:08.845999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:08.845999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:08.845999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:08.845999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:08.845999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:08.845999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:08.845999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:08.845999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:08.845999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:08.845999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:08.845999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:08.845999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:08.845999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 282/544] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:08.852113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00fa95-02fc-4f2a-a701-35452b25fa78 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:08.978900 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c595f77-e322-450f-a762-7d6b2b421b1f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:08.979235 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c595f77-e322-450f-a762-7d6b2b421b1f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:08.979580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c595f77-e322-450f-a762-7d6b2b421b1f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:08.981209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 283/545] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:54:08.988602 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83120bc5-b1ba-4c28-991d-850b9da6b609 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:08.989109 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83120bc5-b1ba-4c28-991d-850b9da6b609 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:08.989615 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83120bc5-b1ba-4c28-991d-850b9da6b609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:09.013139 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00fa95-02fc-4f2a-a701-35452b25fa78 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:09.028845 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-83120bc5-b1ba-4c28-991d-850b9da6b609 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:09.030060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-83120bc5-b1ba-4c28-991d-850b9da6b609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:09.086435 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89cb9f-b1cc-4725-a10c-d55dfdc918dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:09.087974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89cb9f-b1cc-4725-a10c-d55dfdc918dd None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:09.088634 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89cb9f-b1cc-4725-a10c-d55dfdc918dd None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:09.089656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89cb9f-b1cc-4725-a10c-d55dfdc918dd None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:09.091066 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89cb9f-b1cc-4725-a10c-d55dfdc918dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:09.092034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89cb9f-b1cc-4725-a10c-d55dfdc918dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:09.093990 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89cb9f-b1cc-4725-a10c-d55dfdc918dd None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:09.110980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 263/546] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:09.117137 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89cb9f-b1cc-4725-a10c-d55dfdc918dd None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:09.176248 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 284/547] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:54:09.221328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:09.279183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89cb9f-b1cc-4725-a10c-d55dfdc918dd None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:09.322889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663dbe9-e811-4865-abbf-b5159f15ec37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:09.323552 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663dbe9-e811-4865-abbf-b5159f15ec37 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:09.323818 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663dbe9-e811-4865-abbf-b5159f15ec37 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:09.324089 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663dbe9-e811-4865-abbf-b5159f15ec37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:09.340114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663dbe9-e811-4865-abbf-b5159f15ec37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:09.341652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663dbe9-e811-4865-abbf-b5159f15ec37 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:09.361186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-c663dbe9-e811-4865-abbf-b5159f15ec37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:09.361186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:09.361186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:09.361186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:09.361186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:09.361186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:09.361186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:09.361186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:09.361186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:09.361186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:09.361186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:09.361186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:09.361186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:09.361186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:09.361186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:09.361186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:09.362113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:09.363484 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:09.363484 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:09.363484 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:09.367053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 264/548] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:09.368273 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83120bc5-b1ba-4c28-991d-850b9da6b609 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:09.462691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:09.523701 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83120bc5-b1ba-4c28-991d-850b9da6b609 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:09.569765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24a64e-7f6a-49c4-8dd0-ff294f70900a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:09.570359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24a64e-7f6a-49c4-8dd0-ff294f70900a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:09.570640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24a64e-7f6a-49c4-8dd0-ff294f70900a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:09.570913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24a64e-7f6a-49c4-8dd0-ff294f70900a None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:09.571593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24a64e-7f6a-49c4-8dd0-ff294f70900a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:09.571895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24a64e-7f6a-49c4-8dd0-ff294f70900a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:09.573279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24a64e-7f6a-49c4-8dd0-ff294f70900a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:09.591673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 285/549] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:09.597093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663dbe9-e811-4865-abbf-b5159f15ec37 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:09.661121 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24a64e-7f6a-49c4-8dd0-ff294f70900a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:09.787089 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663dbe9-e811-4865-abbf-b5159f15ec37 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:09.807250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:09.855899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24a64e-7f6a-49c4-8dd0-ff294f70900a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:09.859174 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572fd87-872e-4ec6-a874-833a0de466df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:09.859830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572fd87-872e-4ec6-a874-833a0de466df None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:09.860110 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572fd87-872e-4ec6-a874-833a0de466df None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:09.860381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572fd87-872e-4ec6-a874-833a0de466df None admin] PATH_INFO: `/v3/users/swift` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:09.876452 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572fd87-872e-4ec6-a874-833a0de466df None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:09.878005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572fd87-872e-4ec6-a874-833a0de466df None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:09.891257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-1572fd87-872e-4ec6-a874-833a0de466df None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 07:54:09.891257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:09.891257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:09.891257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:09.891257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:09.891257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:09.891257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:09.891257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:09.891257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:09.891257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:09.891257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:09.891257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:09.891257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:09.891257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:09.891257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:09.891257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:09.892370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:09.893518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:09.893518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:09.893518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:09.893518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:09.893518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:09.893518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:09.893518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:09.893518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:09.893518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 07:54:09.893518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:09.894762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:09.897187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 265/550] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:54:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:09.903762 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572fd87-872e-4ec6-a874-833a0de466df None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:09.945219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac20c9-7e51-445e-ba49-89c8d0ae7bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:09.945853 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac20c9-7e51-445e-ba49-89c8d0ae7bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:09.946132 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac20c9-7e51-445e-ba49-89c8d0ae7bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:09.946400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac20c9-7e51-445e-ba49-89c8d0ae7bc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:09.972544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac20c9-7e51-445e-ba49-89c8d0ae7bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:09.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-92ac20c9-7e51-445e-ba49-89c8d0ae7bc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:09.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:09.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:09.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:09.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:09.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:09.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:09.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:09.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:09.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:09.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:09.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:09.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:09.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:09.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:09.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:09.982288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:09.982288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:09.982288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:09.982288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:09.982288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:09.982288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:09.982288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:09.982288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:09.982288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:09.982288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:09.982288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:09.982288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:09.982288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:09.982288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:09.982288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:09.982288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:09.983160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:09.983160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:09.983160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:09.983160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:09.983160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:09.983160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:09.983160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:09.983160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:09.983160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:09.983160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:09.983160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:09.983160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:09.983160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 286/551] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:09.983160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4695ff-dbd0-48cb-bc5d-31be7cb0ccf4 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:09.983898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4695ff-dbd0-48cb-bc5d-31be7cb0ccf4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:09.983898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4695ff-dbd0-48cb-bc5d-31be7cb0ccf4 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:09.983898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 287/552] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:09.993517 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4695ff-dbd0-48cb-bc5d-31be7cb0ccf4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:10.057483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572fd87-872e-4ec6-a874-833a0de466df None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:10.104091 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37152778-794c-47cd-b09e-fa023db12309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:10.104703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37152778-794c-47cd-b09e-fa023db12309 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:10.104983 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37152778-794c-47cd-b09e-fa023db12309 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:10.105255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37152778-794c-47cd-b09e-fa023db12309 None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:10.106312 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37152778-794c-47cd-b09e-fa023db12309 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:10.106566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37152778-794c-47cd-b09e-fa023db12309 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:10.108580 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37152778-794c-47cd-b09e-fa023db12309 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:10.127745 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4695ff-dbd0-48cb-bc5d-31be7cb0ccf4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:10.129048 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 266/553] 10.222.0.30 () {56 vars in 1135 bytes} [Thu Oct 6 07:54:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:10.130588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18751cb-6123-4f9d-8614-a621cdca620a None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:10.131087 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18751cb-6123-4f9d-8614-a621cdca620a None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:10.131577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18751cb-6123-4f9d-8614-a621cdca620a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:10.151876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-b18751cb-6123-4f9d-8614-a621cdca620a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:10.153368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b18751cb-6123-4f9d-8614-a621cdca620a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:10.172954 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f60095-5edc-4249-9d76-5cff9a037e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:10.173564 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f60095-5edc-4249-9d76-5cff9a037e39 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:10.173844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f60095-5edc-4249-9d76-5cff9a037e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:10.174118 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f60095-5edc-4249-9d76-5cff9a037e39 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:10.174837 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f60095-5edc-4249-9d76-5cff9a037e39 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:10.175172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f60095-5edc-4249-9d76-5cff9a037e39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:10.176547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f60095-5edc-4249-9d76-5cff9a037e39 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:10.180126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:10.190810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 288/554] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:10.190810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f60095-5edc-4249-9d76-5cff9a037e39 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:10.226628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:10.234698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 267/555] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:10.235855 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18751cb-6123-4f9d-8614-a621cdca620a None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:10.300260 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f60095-5edc-4249-9d76-5cff9a037e39 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:10.338684 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d96c6b-cd92-489a-b991-11a0a470afc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:10.339322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d96c6b-cd92-489a-b991-11a0a470afc6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:10.339601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d96c6b-cd92-489a-b991-11a0a470afc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:10.339875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d96c6b-cd92-489a-b991-11a0a470afc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:10.414550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18751cb-6123-4f9d-8614-a621cdca620a None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:10.428061 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d96c6b-cd92-489a-b991-11a0a470afc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:10.428061 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d96c6b-cd92-489a-b991-11a0a470afc6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:10.463399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:10.515652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428ccbf-bc52-4776-b8d5-f8cee63f3122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:10.516300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428ccbf-bc52-4776-b8d5-f8cee63f3122 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:10.516565 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428ccbf-bc52-4776-b8d5-f8cee63f3122 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:10.516838 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428ccbf-bc52-4776-b8d5-f8cee63f3122 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:10.542044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 289/556] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:54:10.547468 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d96c6b-cd92-489a-b991-11a0a470afc6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:10.547776 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428ccbf-bc52-4776-b8d5-f8cee63f3122 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:10.549257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428ccbf-bc52-4776-b8d5-f8cee63f3122 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:10.580333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-c428ccbf-bc52-4776-b8d5-f8cee63f3122 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:10.580333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:10.580333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:10.580333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:10.580333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:10.580333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:10.580333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:10.580333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:10.580333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:10.580333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:10.580333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:10.580333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:10.580333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:10.580333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:10.580333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:10.580333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:10.581326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:10.582205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:10.582205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:10.582205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:10.591320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 268/557] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:10.592258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428ccbf-bc52-4776-b8d5-f8cee63f3122 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:10.614483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:10.665112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d96c6b-cd92-489a-b991-11a0a470afc6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:10.713002 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671062c-cb61-4374-a5a1-c84f4d5ab4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:10.714131 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671062c-cb61-4374-a5a1-c84f4d5ab4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:10.714698 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671062c-cb61-4374-a5a1-c84f4d5ab4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:10.715370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671062c-cb61-4374-a5a1-c84f4d5ab4e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:10.717849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:10.729736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671062c-cb61-4374-a5a1-c84f4d5ab4e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:10.734201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671062c-cb61-4374-a5a1-c84f4d5ab4e5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:10.747652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a671062c-cb61-4374-a5a1-c84f4d5ab4e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:10.747652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:10.747652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:10.747652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:10.747652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:10.747652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:10.747652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:10.747652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:10.747652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:10.747652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:10.747652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:10.747652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:10.747652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:10.747652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:10.747652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:10.747652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:10.749332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:10.750200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:10.750200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:10.750200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:10.753886 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 290/558] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:10.754707 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671062c-cb61-4374-a5a1-c84f4d5ab4e5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:10.773522 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428ccbf-bc52-4776-b8d5-f8cee63f3122 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:10.812057 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e290e-1250-4aaf-8747-2ba93fae6491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:10.812665 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e290e-1250-4aaf-8747-2ba93fae6491 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:10.812938 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e290e-1250-4aaf-8747-2ba93fae6491 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:10.813202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e290e-1250-4aaf-8747-2ba93fae6491 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:10.813835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e290e-1250-4aaf-8747-2ba93fae6491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:10.814131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e290e-1250-4aaf-8747-2ba93fae6491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:10.815553 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e290e-1250-4aaf-8747-2ba93fae6491 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:10.828634 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 269/559] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:10.828634 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e290e-1250-4aaf-8747-2ba93fae6491 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:10.870470 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671062c-cb61-4374-a5a1-c84f4d5ab4e5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:10.903576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:10.906382 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc5cd1-59b2-4f7b-8dc3-2a4329fc9cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:10.907043 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc5cd1-59b2-4f7b-8dc3-2a4329fc9cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:10.907326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc5cd1-59b2-4f7b-8dc3-2a4329fc9cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:10.907597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc5cd1-59b2-4f7b-8dc3-2a4329fc9cf2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:10.919929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc5cd1-59b2-4f7b-8dc3-2a4329fc9cf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:10.925361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6dcc5cd1-59b2-4f7b-8dc3-2a4329fc9cf2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:10.925361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:10.925361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:10.925361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:10.925361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:10.925361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:10.925361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:10.925361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:10.925361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:10.925361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:10.925361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:10.925361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:10.925361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:10.925361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:10.925361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:10.925361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:10.926384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:10.926384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:10.926384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:10.926384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:10.926384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:10.926384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:10.926384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:10.926384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:10.926384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:10.926384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:10.926384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:10.926384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:10.926384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:10.926384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:10.926384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:10.926384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:10.927890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:10.927890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:10.927890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:10.927890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:10.927890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:10.927890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:10.927890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:10.927890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:10.927890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:10.927890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:10.927890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:10.927890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:10.927890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 291/560] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:10.929288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5e99f-370e-4730-821a-918beb6f5f62 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:10.929288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5e99f-370e-4730-821a-918beb6f5f62 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:10.929288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5e99f-370e-4730-821a-918beb6f5f62 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:10.935314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 292/561] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:10.935314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5e99f-370e-4730-821a-918beb6f5f62 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:10.941768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e290e-1250-4aaf-8747-2ba93fae6491 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:10.984752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511be8c9-b98e-4a45-b611-aa887023ee82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:10.985351 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511be8c9-b98e-4a45-b611-aa887023ee82 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:10.985621 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511be8c9-b98e-4a45-b611-aa887023ee82 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:10.985886 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511be8c9-b98e-4a45-b611-aa887023ee82 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:10.986522 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511be8c9-b98e-4a45-b611-aa887023ee82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:10.986814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511be8c9-b98e-4a45-b611-aa887023ee82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:10.988227 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511be8c9-b98e-4a45-b611-aa887023ee82 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:10.999348 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:11.000911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 270/562] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:11.001799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511be8c9-b98e-4a45-b611-aa887023ee82 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:11.060042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5e99f-370e-4730-821a-918beb6f5f62 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:11.109407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511be8c9-b98e-4a45-b611-aa887023ee82 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:11.144737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:11.150702 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a5ba7-1c23-453f-aa13-e9bea34b385f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:11.151347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a5ba7-1c23-453f-aa13-e9bea34b385f None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:11.151462 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3660bf-36ee-42da-aab2-7900b9534ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:11.151637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a5ba7-1c23-453f-aa13-e9bea34b385f None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:11.151901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a5ba7-1c23-453f-aa13-e9bea34b385f None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:11.152009 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3660bf-36ee-42da-aab2-7900b9534ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:11.152277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3660bf-36ee-42da-aab2-7900b9534ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:11.152610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3660bf-36ee-42da-aab2-7900b9534ea8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:11.152699 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a5ba7-1c23-453f-aa13-e9bea34b385f None admin] RBAC: Adding query filter params (name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:11.152856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a5ba7-1c23-453f-aa13-e9bea34b385f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:11.154255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a5ba7-1c23-453f-aa13-e9bea34b385f None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:11.159929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3660bf-36ee-42da-aab2-7900b9534ea8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:11.161349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3660bf-36ee-42da-aab2-7900b9534ea8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:11.162660 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 271/563] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:11.164590 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3605efc-79d9-4785-8e6c-1989462ecd90 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:11.164863 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3605efc-79d9-4785-8e6c-1989462ecd90 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:11.165154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3605efc-79d9-4785-8e6c-1989462ecd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:11.178518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-9d3660bf-36ee-42da-aab2-7900b9534ea8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:54:11.178518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:11.178518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:11.178518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:11.178518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:11.178518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:11.178518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:11.178518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:11.178518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:11.178518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:11.178518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:11.178518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:11.178518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:11.178518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:11.178518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:11.178518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:11.184720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:11.185694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:11.185694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:11.185694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:11.185694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:11.185694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:11.185694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:11.185694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:11.185694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:11.185694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:54:11.185694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:11.185694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 293/564] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Oct 6 07:54:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:11.185694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3660bf-36ee-42da-aab2-7900b9534ea8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:11.186368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-e3605efc-79d9-4785-8e6c-1989462ecd90 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:11.186368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e3605efc-79d9-4785-8e6c-1989462ecd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:11.251233 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 272/565] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:11.252846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f37088-6fb0-4ebd-a6e4-8286e672ff1d None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:11.253136 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f37088-6fb0-4ebd-a6e4-8286e672ff1d None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:11.253409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f37088-6fb0-4ebd-a6e4-8286e672ff1d None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:11.254795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 273/566] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:11.255468 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f37088-6fb0-4ebd-a6e4-8286e672ff1d None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:11.283700 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3660bf-36ee-42da-aab2-7900b9534ea8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:11.291350 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:11.318770 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83868f5-6cad-4bf5-b9df-1bd51ef7f9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:11.320142 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83868f5-6cad-4bf5-b9df-1bd51ef7f9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:11.320421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83868f5-6cad-4bf5-b9df-1bd51ef7f9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:11.320690 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83868f5-6cad-4bf5-b9df-1bd51ef7f9a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:11.327987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83868f5-6cad-4bf5-b9df-1bd51ef7f9a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:11.329835 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83868f5-6cad-4bf5-b9df-1bd51ef7f9a4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:11.342535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c83868f5-6cad-4bf5-b9df-1bd51ef7f9a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:11.342535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:11.342535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:11.342535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:11.342535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:11.342535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:11.342535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:11.342535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:11.342535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:11.342535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:11.342535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:11.342535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:11.342535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:11.342535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:11.342535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:11.343490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:11.344379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:11.344379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:11.344379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:11.344379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:11.344379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:11.344379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:11.344379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:11.344379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:11.344379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 294/567] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:11.344859 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83868f5-6cad-4bf5-b9df-1bd51ef7f9a4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:11.381326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f37088-6fb0-4ebd-a6e4-8286e672ff1d None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:11.404823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:11.416740 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea2316-0f69-42be-befc-8c76ff3e7223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:11.420643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea2316-0f69-42be-befc-8c76ff3e7223 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:11.420643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea2316-0f69-42be-befc-8c76ff3e7223 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:11.420643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea2316-0f69-42be-befc-8c76ff3e7223 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:11.428093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea2316-0f69-42be-befc-8c76ff3e7223 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:11.430261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea2316-0f69-42be-befc-8c76ff3e7223 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:11.446411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-d6ea2316-0f69-42be-befc-8c76ff3e7223 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:11.446411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:11.446411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:11.446411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:11.446411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:11.446411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:11.446411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:11.446411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:11.446411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:11.446411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:11.446411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:11.446411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:11.446411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:11.446411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:11.446411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:11.446411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:11.447416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:11.448332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:11.448332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:11.448332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:11.448332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 274/568] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:11.449267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd35fc3-857a-4385-8929-379b9ca85781 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:11.449540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd35fc3-857a-4385-8929-379b9ca85781 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:11.449808 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd35fc3-857a-4385-8929-379b9ca85781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:11.456538 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83868f5-6cad-4bf5-b9df-1bd51ef7f9a4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:11.468517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-2fd35fc3-857a-4385-8929-379b9ca85781 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:11.469698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-2fd35fc3-857a-4385-8929-379b9ca85781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:11.494274 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0104b11b-d1af-4975-94c0-01b9ff0b68f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:11.494880 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104b11b-d1af-4975-94c0-01b9ff0b68f6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:11.495181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104b11b-d1af-4975-94c0-01b9ff0b68f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:11.495447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104b11b-d1af-4975-94c0-01b9ff0b68f6 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:11.496167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104b11b-d1af-4975-94c0-01b9ff0b68f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:11.496465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104b11b-d1af-4975-94c0-01b9ff0b68f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:11.498925 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0104b11b-d1af-4975-94c0-01b9ff0b68f6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:11.523087 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 295/569] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Oct 6 07:54:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:11.523941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0104b11b-d1af-4975-94c0-01b9ff0b68f6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:11.544286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 275/570] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:11.545140 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd35fc3-857a-4385-8929-379b9ca85781 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:11.549942 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:11.628227 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0104b11b-d1af-4975-94c0-01b9ff0b68f6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:11.648549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd35fc3-857a-4385-8929-379b9ca85781 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:11.651495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:11.663949 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee6001-0d26-4be2-b573-87f75dbfbbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:11.664544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee6001-0d26-4be2-b573-87f75dbfbbad None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:11.664820 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee6001-0d26-4be2-b573-87f75dbfbbad None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:11.665085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee6001-0d26-4be2-b573-87f75dbfbbad None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:11.665774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee6001-0d26-4be2-b573-87f75dbfbbad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:11.666077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee6001-0d26-4be2-b573-87f75dbfbbad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:11.667513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee6001-0d26-4be2-b573-87f75dbfbbad None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:11.679950 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 296/571] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:11.680837 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee6001-0d26-4be2-b573-87f75dbfbbad None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:11.690838 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328dae4e-455b-41cc-8ce1-dc6bb6dc9ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:11.691920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dae4e-455b-41cc-8ce1-dc6bb6dc9ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:11.692911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dae4e-455b-41cc-8ce1-dc6bb6dc9ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:11.693196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dae4e-455b-41cc-8ce1-dc6bb6dc9ad1 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:11.694353 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328dae4e-455b-41cc-8ce1-dc6bb6dc9ad1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:11.695034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328dae4e-455b-41cc-8ce1-dc6bb6dc9ad1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:11.696583 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328dae4e-455b-41cc-8ce1-dc6bb6dc9ad1 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:11.716455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 276/572] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:11.716455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328dae4e-455b-41cc-8ce1-dc6bb6dc9ad1 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:11.768034 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:11.786006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee6001-0d26-4be2-b573-87f75dbfbbad None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:11.822467 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f59d4d-981c-4845-ad51-0be06ca8c6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:11.823117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f59d4d-981c-4845-ad51-0be06ca8c6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:11.823389 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f59d4d-981c-4845-ad51-0be06ca8c6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:11.823657 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f59d4d-981c-4845-ad51-0be06ca8c6d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:11.823900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328dae4e-455b-41cc-8ce1-dc6bb6dc9ad1 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:11.833803 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f59d4d-981c-4845-ad51-0be06ca8c6d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:11.835876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f59d4d-981c-4845-ad51-0be06ca8c6d4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:11.851164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-33f59d4d-981c-4845-ad51-0be06ca8c6d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:11.851164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:11.851164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:11.851164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:11.851164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:11.851164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:11.851164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:11.851164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:11.851164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:11.851164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:11.851164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:11.851164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:11.851164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:11.851164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:11.851164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:11.851164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:11.852332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:11.853253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:11.853253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:11.853253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:11.853253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 297/573] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:11.853253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f59d4d-981c-4845-ad51-0be06ca8c6d4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:11.859799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2918e-1acb-4498-a527-2b5737a94807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:11.860419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2918e-1acb-4498-a527-2b5737a94807 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:11.860692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2918e-1acb-4498-a527-2b5737a94807 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:11.861080 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2918e-1acb-4498-a527-2b5737a94807 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:11.871795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e2918e-1acb-4498-a527-2b5737a94807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:11.877135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-77e2918e-1acb-4498-a527-2b5737a94807 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:11.877135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:11.877135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:11.877135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:11.877135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:11.877135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:11.877135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:11.877135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:11.877135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:11.877135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:11.877135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:11.877135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:11.877135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:11.877135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:11.877135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:11.877135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:11.878231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:11.878231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:11.878231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:11.878231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:11.878231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:11.878231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:11.878231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:11.878231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:11.878231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:11.878231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:11.878231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:11.878231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:11.878231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:11.878231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:11.878231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:11.878231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:11.879293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:11.879293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:11.879293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:11.879293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:11.879293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:11.879293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:11.879293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:11.879293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:11.879293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:11.879293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:11.879293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:11.879293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:11.879293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 277/574] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:11.879293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2918e-1acb-4498-a527-2b5737a94807 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:11.905198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:11.957009 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f59d4d-981c-4845-ad51-0be06ca8c6d4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:11.977959 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2918e-1acb-4498-a527-2b5737a94807 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:11.989942 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91994e-67a4-42b4-93ae-6147714de870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:11.990533 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91994e-67a4-42b4-93ae-6147714de870 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:11.990812 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91994e-67a4-42b4-93ae-6147714de870 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:11.991703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91994e-67a4-42b4-93ae-6147714de870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:11.992410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91994e-67a4-42b4-93ae-6147714de870 None admin] RBAC: Adding query filter params (role.id=9f0e36c2ed8843fbabd3a4cf30bc3725, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=3fa1c430c9f7416eaf1d17300b827e05) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:11.992709 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91994e-67a4-42b4-93ae-6147714de870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:11.994127 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91994e-67a4-42b4-93ae-6147714de870 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:11.996934 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:12.000144 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 298/575] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:11 2022] GET /identity/v3/role_assignments?user.id=3fa1c430c9f7416eaf1d17300b827e05&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:12.000923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91994e-67a4-42b4-93ae-6147714de870 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:12.019951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506d512-1c4f-4211-abbc-ab57cc7a73fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:12.020595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506d512-1c4f-4211-abbc-ab57cc7a73fb None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:12.020868 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506d512-1c4f-4211-abbc-ab57cc7a73fb None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:12.021133 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506d512-1c4f-4211-abbc-ab57cc7a73fb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:12.030463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506d512-1c4f-4211-abbc-ab57cc7a73fb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:12.032440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506d512-1c4f-4211-abbc-ab57cc7a73fb None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:12.044980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-7506d512-1c4f-4211-abbc-ab57cc7a73fb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 07:54:12.044980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:12.044980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:12.044980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:12.044980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:12.044980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:12.044980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:12.044980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:12.044980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:12.044980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:12.044980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:12.044980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:12.044980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:12.044980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:12.044980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:12.044980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:12.046203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:12.047823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:12.047823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:12.047823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:12.047823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:12.047823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:12.047823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:12.047823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:12.047823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:12.047823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 07:54:12.047823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:12.047823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 278/576] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:54:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:12.047823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506d512-1c4f-4211-abbc-ab57cc7a73fb None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:12.154878 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91994e-67a4-42b4-93ae-6147714de870 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:12.174907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:12.192748 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506d512-1c4f-4211-abbc-ab57cc7a73fb None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:12.195274 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab4588-354a-44c7-b740-e5debd5e1c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:12.196777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab4588-354a-44c7-b740-e5debd5e1c3b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:12.199474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab4588-354a-44c7-b740-e5debd5e1c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:12.199474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab4588-354a-44c7-b740-e5debd5e1c3b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:12.208361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab4588-354a-44c7-b740-e5debd5e1c3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:12.214283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-86ab4588-354a-44c7-b740-e5debd5e1c3b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:12.214283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:12.214283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:12.214283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:12.214283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:12.214283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:12.214283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:12.214283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:12.214283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:12.214283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:12.214283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:12.214283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:12.214283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:12.214283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:12.214283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:12.214283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:12.215454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:12.215454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:12.215454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:12.215454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:12.215454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:12.215454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:12.215454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:12.215454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:12.215454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:12.215454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:12.215454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:12.215454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:12.215454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:12.215454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:12.215454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:12.215454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:12.216695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:12.216695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:12.216695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:12.216695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:12.216695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:12.216695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:12.216695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:12.216695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:12.216695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:12.216695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:12.216695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:12.216695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:12.216695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 299/577] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:12.216695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab4588-354a-44c7-b740-e5debd5e1c3b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:12.236242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a1249-8c12-430e-af54-b370656ade73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:12.236856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a1249-8c12-430e-af54-b370656ade73 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:12.237132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a1249-8c12-430e-af54-b370656ade73 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:12.237403 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a1249-8c12-430e-af54-b370656ade73 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:12.238045 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a1249-8c12-430e-af54-b370656ade73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:12.238347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a1249-8c12-430e-af54-b370656ade73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:12.239778 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a1249-8c12-430e-af54-b370656ade73 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:12.253418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 279/578] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:12.254519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a1249-8c12-430e-af54-b370656ade73 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:12.285667 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:12.332525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab4588-354a-44c7-b740-e5debd5e1c3b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:12.357010 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a1249-8c12-430e-af54-b370656ade73 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:12.368941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700f18d-e995-4c5f-bc25-927524d0ce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:12.369543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700f18d-e995-4c5f-bc25-927524d0ce1a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:12.369827 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700f18d-e995-4c5f-bc25-927524d0ce1a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:12.370087 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700f18d-e995-4c5f-bc25-927524d0ce1a None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:12.370761 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700f18d-e995-4c5f-bc25-927524d0ce1a None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:12.371104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700f18d-e995-4c5f-bc25-927524d0ce1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:12.372479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700f18d-e995-4c5f-bc25-927524d0ce1a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:12.382081 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 300/579] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:12.383747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700f18d-e995-4c5f-bc25-927524d0ce1a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:12.400673 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea174f2-c7da-4036-9aae-46a98b4ec374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:12.401417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea174f2-c7da-4036-9aae-46a98b4ec374 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:12.401761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea174f2-c7da-4036-9aae-46a98b4ec374 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:12.402089 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea174f2-c7da-4036-9aae-46a98b4ec374 None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:12.402873 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea174f2-c7da-4036-9aae-46a98b4ec374 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:12.403262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea174f2-c7da-4036-9aae-46a98b4ec374 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:12.404740 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea174f2-c7da-4036-9aae-46a98b4ec374 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:12.425782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 280/580] 10.222.0.30 () {56 vars in 1137 bytes} [Thu Oct 6 07:54:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:12.427439 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea174f2-c7da-4036-9aae-46a98b4ec374 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:12.433569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:12.498563 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700f18d-e995-4c5f-bc25-927524d0ce1a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:12.523627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:12.535241 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930923d1-3776-47e0-b323-78249ed92a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:12.541021 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930923d1-3776-47e0-b323-78249ed92a00 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:12.541021 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930923d1-3776-47e0-b323-78249ed92a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:12.541021 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930923d1-3776-47e0-b323-78249ed92a00 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:12.541282 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea174f2-c7da-4036-9aae-46a98b4ec374 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:12.541373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930923d1-3776-47e0-b323-78249ed92a00 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:12.541373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930923d1-3776-47e0-b323-78249ed92a00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:12.541373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930923d1-3776-47e0-b323-78249ed92a00 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:12.548440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 301/581] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:12.549115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930923d1-3776-47e0-b323-78249ed92a00 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:12.578493 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41a046-0b43-4405-9521-07a81e6e1981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:12.579475 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41a046-0b43-4405-9521-07a81e6e1981 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:12.579899 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41a046-0b43-4405-9521-07a81e6e1981 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:12.580169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41a046-0b43-4405-9521-07a81e6e1981 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:12.588192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41a046-0b43-4405-9521-07a81e6e1981 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:12.589642 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41a046-0b43-4405-9521-07a81e6e1981 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:12.601398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-7b41a046-0b43-4405-9521-07a81e6e1981 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:12.601398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:12.601398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:12.601398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:12.601398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:12.601398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:12.601398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:12.601398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:12.601398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:12.601398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:12.601398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:12.601398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:12.601398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:12.601398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:12.601398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:12.602307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:12.603242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:12.603242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:12.603242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:12.603242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:12.603242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:12.603242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:12.603242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:12.603242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:12.603242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 281/582] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:12.603242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41a046-0b43-4405-9521-07a81e6e1981 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:12.663694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930923d1-3776-47e0-b323-78249ed92a00 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:12.676035 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:12.700770 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73aaa2-d241-4bfa-8c02-24eef9d39544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:12.701356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73aaa2-d241-4bfa-8c02-24eef9d39544 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:12.701627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73aaa2-d241-4bfa-8c02-24eef9d39544 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:12.701896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73aaa2-d241-4bfa-8c02-24eef9d39544 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:12.713951 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73aaa2-d241-4bfa-8c02-24eef9d39544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:12.715438 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73aaa2-d241-4bfa-8c02-24eef9d39544 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:12.716802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41a046-0b43-4405-9521-07a81e6e1981 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:12.732731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5c73aaa2-d241-4bfa-8c02-24eef9d39544 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:12.732731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:12.732731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:12.732731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:12.732731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:12.732731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:12.732731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:12.732731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:12.732731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:12.732731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:12.732731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:12.732731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:12.732731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:12.732731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:12.732731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:12.732731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:12.733754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:12.734664 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:12.734664 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:12.734664 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:12.734664 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 302/583] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:12.734664 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73aaa2-d241-4bfa-8c02-24eef9d39544 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:12.757999 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5006af-95ca-432c-9e78-4c16cac08f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:12.758537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5006af-95ca-432c-9e78-4c16cac08f51 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:12.758819 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5006af-95ca-432c-9e78-4c16cac08f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:12.759122 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5006af-95ca-432c-9e78-4c16cac08f51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:12.769894 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5006af-95ca-432c-9e78-4c16cac08f51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:12.771371 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5006af-95ca-432c-9e78-4c16cac08f51 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:12.781875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:12.786050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-cc5006af-95ca-432c-9e78-4c16cac08f51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:12.786050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:12.786050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:12.786050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:12.786050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:12.786050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:12.786050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:12.786050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:12.786050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:12.786050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:12.786050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:12.786050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:12.786050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:12.786050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:12.786050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:12.786050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:12.787209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:12.789171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:12.789171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:12.789171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:12.792326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 282/584] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:12.793083 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5006af-95ca-432c-9e78-4c16cac08f51 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:12.857935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73aaa2-d241-4bfa-8c02-24eef9d39544 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:12.895766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f43e18-7599-4573-aebc-23dcf651b14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:12.896715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f43e18-7599-4573-aebc-23dcf651b14d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:12.897040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f43e18-7599-4573-aebc-23dcf651b14d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:12.897895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f43e18-7599-4573-aebc-23dcf651b14d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:12.900640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5006af-95ca-432c-9e78-4c16cac08f51 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:12.908231 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f43e18-7599-4573-aebc-23dcf651b14d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:12.909693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f43e18-7599-4573-aebc-23dcf651b14d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:12.915166 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:12.924542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d5f43e18-7599-4573-aebc-23dcf651b14d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:12.924542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:12.924542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:12.924542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:12.924542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:12.924542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:12.924542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:12.924542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:12.924542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:12.924542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:12.924542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:12.924542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:12.924542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:12.924542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:12.924542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:12.924542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:12.931325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:12.932425 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:12.932425 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:12.932425 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:12.932425 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 303/585] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:12.932425 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f43e18-7599-4573-aebc-23dcf651b14d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:12.943019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e5f69-3e86-4c96-9e4b-ce1d29a483b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:12.943628 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e5f69-3e86-4c96-9e4b-ce1d29a483b5 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:12.943900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e5f69-3e86-4c96-9e4b-ce1d29a483b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:12.944835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e5f69-3e86-4c96-9e4b-ce1d29a483b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:12.945552 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e5f69-3e86-4c96-9e4b-ce1d29a483b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:12.945967 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e5f69-3e86-4c96-9e4b-ce1d29a483b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:12.947705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e5f69-3e86-4c96-9e4b-ce1d29a483b5 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:12.965967 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 283/586] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:12.965967 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e5f69-3e86-4c96-9e4b-ce1d29a483b5 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:13.030807 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:13.042968 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f43e18-7599-4573-aebc-23dcf651b14d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:13.076865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ecb065-a247-49f9-9b62-262ee29964f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:13.077458 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ecb065-a247-49f9-9b62-262ee29964f5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:13.077731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ecb065-a247-49f9-9b62-262ee29964f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:13.078066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ecb065-a247-49f9-9b62-262ee29964f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:13.078768 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ecb065-a247-49f9-9b62-262ee29964f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:13.079296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ecb065-a247-49f9-9b62-262ee29964f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:13.080652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ecb065-a247-49f9-9b62-262ee29964f5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:13.081774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e5f69-3e86-4c96-9e4b-ce1d29a483b5 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:13.097030 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 304/587] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:13.097030 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ecb065-a247-49f9-9b62-262ee29964f5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:13.121562 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad108eb2-d1ca-4093-9440-42b64fae09fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:13.122168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad108eb2-d1ca-4093-9440-42b64fae09fa None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:13.122443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad108eb2-d1ca-4093-9440-42b64fae09fa None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:13.122705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad108eb2-d1ca-4093-9440-42b64fae09fa None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:13.123620 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad108eb2-d1ca-4093-9440-42b64fae09fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:13.123914 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad108eb2-d1ca-4093-9440-42b64fae09fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:13.125279 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad108eb2-d1ca-4093-9440-42b64fae09fa None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:13.140509 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 284/588] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:13.140509 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad108eb2-d1ca-4093-9440-42b64fae09fa None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:13.174254 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:13.209054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ecb065-a247-49f9-9b62-262ee29964f5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:13.241615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad108eb2-d1ca-4093-9440-42b64fae09fa None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:13.244197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff6714-bf03-4113-8df1-079a4b841ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:13.244803 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff6714-bf03-4113-8df1-079a4b841ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:13.245081 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff6714-bf03-4113-8df1-079a4b841ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:13.245354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff6714-bf03-4113-8df1-079a4b841ad4 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:13.245997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff6714-bf03-4113-8df1-079a4b841ad4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:13.246299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff6714-bf03-4113-8df1-079a4b841ad4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:13.247763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff6714-bf03-4113-8df1-079a4b841ad4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:13.260833 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 305/589] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:13.261662 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff6714-bf03-4113-8df1-079a4b841ad4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:13.279566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339eec4c-7773-4871-8b9f-3c7bcd9ab7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:13.280221 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339eec4c-7773-4871-8b9f-3c7bcd9ab7f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:13.280495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339eec4c-7773-4871-8b9f-3c7bcd9ab7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:13.280965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339eec4c-7773-4871-8b9f-3c7bcd9ab7f8 None admin] PATH_INFO: `/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/209795285384486183cda455fe316bc9/roles/9f0e36c2ed8843fbabd3a4cf30bc3725` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:13.297330 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339eec4c-7773-4871-8b9f-3c7bcd9ab7f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=399f0095cc8b4c64893ccc75af13c52d, user_id=209795285384486183cda455fe316bc9, role_id=9f0e36c2ed8843fbabd3a4cf30bc3725)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:13.298808 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339eec4c-7773-4871-8b9f-3c7bcd9ab7f8 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:13.323334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:13.369286 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff6714-bf03-4113-8df1-079a4b841ad4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:13.403053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f259459-c888-4979-8ed9-5ca9942c9c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:13.403660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f259459-c888-4979-8ed9-5ca9942c9c4d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:13.416524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f259459-c888-4979-8ed9-5ca9942c9c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:13.416524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f259459-c888-4979-8ed9-5ca9942c9c4d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:13.416524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f259459-c888-4979-8ed9-5ca9942c9c4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:13.416524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f259459-c888-4979-8ed9-5ca9942c9c4d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:13.423819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6f259459-c888-4979-8ed9-5ca9942c9c4d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 06 07:54:13.423819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:13.423819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:13.423819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:13.423819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:13.423819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:13.423819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:13.423819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:13.423819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:13.423819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:13.423819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:13.423819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:13.423819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:13.423819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:13.423819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:13.423819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:13.424947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:13.425976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:13.425976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:13.425976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:13.425976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:13.425976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:13.425976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:13.425976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:13.425976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:13.425976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 06 07:54:13.425976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:13.425976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 306/590] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:54:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:13.426603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f259459-c888-4979-8ed9-5ca9942c9c4d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:13.453466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 285/591] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:54:13 2022] PUT /identity/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/209795285384486183cda455fe316bc9/roles/9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:54:13.455012 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339eec4c-7773-4871-8b9f-3c7bcd9ab7f8 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:13.571351 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:13.587108 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:13.605340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f259459-c888-4979-8ed9-5ca9942c9c4d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:13.606662 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339eec4c-7773-4871-8b9f-3c7bcd9ab7f8 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:13.657260 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc97da-1e1d-422b-be25-5a78e1d83cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:13.664404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98e14a-b8f4-4c29-9898-edffa1b19c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:13.664566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc97da-1e1d-422b-be25-5a78e1d83cfe None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:13.664566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc97da-1e1d-422b-be25-5a78e1d83cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:13.664566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc97da-1e1d-422b-be25-5a78e1d83cfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:13.664801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98e14a-b8f4-4c29-9898-edffa1b19c5e None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:13.664801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98e14a-b8f4-4c29-9898-edffa1b19c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:13.664801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98e14a-b8f4-4c29-9898-edffa1b19c5e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:13.673654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc97da-1e1d-422b-be25-5a78e1d83cfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:13.675480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc97da-1e1d-422b-be25-5a78e1d83cfe None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:13.676125 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98e14a-b8f4-4c29-9898-edffa1b19c5e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:13.678105 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98e14a-b8f4-4c29-9898-edffa1b19c5e None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:13.691275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e4bc97da-1e1d-422b-be25-5a78e1d83cfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:13.691275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:13.691275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:13.691275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:13.691275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:13.691275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:13.691275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:13.691275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:13.691275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:13.691275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:13.691275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:13.691275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:13.691275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:13.691275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:13.691275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:13.692599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:13.693495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:13.693495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:13.693495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:13.693495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:13.693495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:13.693495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:13.693495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:13.693495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:13.693495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 307/592] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:13.697848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-bb98e14a-b8f4-4c29-9898-edffa1b19c5e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 07:54:13.697848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:13.697848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:13.697848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:13.697848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:13.697848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:13.697848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:13.697848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:13.697848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:13.697848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:13.697848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:13.697848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:13.697848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:13.697848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:13.697848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:13.697848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:13.698989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:13.700176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:13.700176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:13.700176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:13.700176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:13.700176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:13.700176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:13.700176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:13.700176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:13.700176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 07:54:13.700176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:13.701126 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc97da-1e1d-422b-be25-5a78e1d83cfe None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:13.701257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 286/593] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:13.705874 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98e14a-b8f4-4c29-9898-edffa1b19c5e None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:13.848223 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98e14a-b8f4-4c29-9898-edffa1b19c5e None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:13.860006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc97da-1e1d-422b-be25-5a78e1d83cfe None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:13.903106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:13.907436 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24645383-9c08-44eb-876e-c6390826eeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:13.908180 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24645383-9c08-44eb-876e-c6390826eeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:13.909031 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24645383-9c08-44eb-876e-c6390826eeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:13.909404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24645383-9c08-44eb-876e-c6390826eeb0 None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:13.910408 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24645383-9c08-44eb-876e-c6390826eeb0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:13.910961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24645383-9c08-44eb-876e-c6390826eeb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:13.914544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade5a0d-2139-4ed2-8ac7-e7440ca89384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:13.914748 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24645383-9c08-44eb-876e-c6390826eeb0 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:13.914913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade5a0d-2139-4ed2-8ac7-e7440ca89384 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:13.915235 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade5a0d-2139-4ed2-8ac7-e7440ca89384 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:13.916354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade5a0d-2139-4ed2-8ac7-e7440ca89384 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:13.918776 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade5a0d-2139-4ed2-8ac7-e7440ca89384 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:13.919324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade5a0d-2139-4ed2-8ac7-e7440ca89384 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:13.921010 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade5a0d-2139-4ed2-8ac7-e7440ca89384 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:13.940210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 287/594] 10.222.0.30 () {56 vars in 1139 bytes} [Thu Oct 6 07:54:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:13.940210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24645383-9c08-44eb-876e-c6390826eeb0 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:13.941192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:13.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 308/595] 10.222.0.30 () {56 vars in 1137 bytes} [Thu Oct 6 07:54:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:13.963393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade5a0d-2139-4ed2-8ac7-e7440ca89384 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:14.087800 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24645383-9c08-44eb-876e-c6390826eeb0 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:14.111206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade5a0d-2139-4ed2-8ac7-e7440ca89384 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:14.133954 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea8cc5-cf79-409b-befe-060873e0c839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:14.134568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea8cc5-cf79-409b-befe-060873e0c839 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:14.134863 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea8cc5-cf79-409b-befe-060873e0c839 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:14.135174 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea8cc5-cf79-409b-befe-060873e0c839 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:14.135830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea8cc5-cf79-409b-befe-060873e0c839 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:14.136128 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea8cc5-cf79-409b-befe-060873e0c839 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:14.139443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea8cc5-cf79-409b-befe-060873e0c839 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:14.153127 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e079f-8cbb-459e-b099-b5c9882954f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:14.153921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e079f-8cbb-459e-b099-b5c9882954f9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:14.154299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e079f-8cbb-459e-b099-b5c9882954f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:14.154771 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e079f-8cbb-459e-b099-b5c9882954f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:14.160917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 288/596] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:14.160917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea8cc5-cf79-409b-befe-060873e0c839 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:14.169822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e079f-8cbb-459e-b099-b5c9882954f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:14.171368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e079f-8cbb-459e-b099-b5c9882954f9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:14.192346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5a1e079f-8cbb-459e-b099-b5c9882954f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:14.192346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:14.192346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:14.192346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:14.192346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:14.192346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:14.192346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:14.192346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:14.192346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:14.192346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:14.192346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:14.192346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:14.192346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:14.192346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:14.192346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:14.192346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:14.193557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:14.195660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:14.195660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:14.195660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:14.195660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 309/597] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:14.195660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e079f-8cbb-459e-b099-b5c9882954f9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:14.241476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:14.249786 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:14.296748 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea8cc5-cf79-409b-befe-060873e0c839 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:14.321686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e079f-8cbb-459e-b099-b5c9882954f9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:14.345960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fb944-b325-49f6-85f1-9bd400ec3a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:14.346853 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fb944-b325-49f6-85f1-9bd400ec3a30 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:14.347322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fb944-b325-49f6-85f1-9bd400ec3a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:14.347829 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fb944-b325-49f6-85f1-9bd400ec3a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:14.363813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0fb944-b325-49f6-85f1-9bd400ec3a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:14.364865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0704e54-3de0-4f15-afd3-a923af5e6b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:14.365594 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0704e54-3de0-4f15-afd3-a923af5e6b3c None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:14.366419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0fb944-b325-49f6-85f1-9bd400ec3a30 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:14.366566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0704e54-3de0-4f15-afd3-a923af5e6b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:14.366713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0704e54-3de0-4f15-afd3-a923af5e6b3c None admin] PATH_INFO: `/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/a899c6dabca948269f6c1370425f244b/roles/9f0e36c2ed8843fbabd3a4cf30bc3725` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:14.385889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0704e54-3de0-4f15-afd3-a923af5e6b3c None admin] RBAC: Authorizing `identity:create_grant(project_id=399f0095cc8b4c64893ccc75af13c52d, user_id=a899c6dabca948269f6c1370425f244b, role_id=9f0e36c2ed8843fbabd3a4cf30bc3725)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:14.388342 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0704e54-3de0-4f15-afd3-a923af5e6b3c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:14.392588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-0a0fb944-b325-49f6-85f1-9bd400ec3a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:14.392588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:14.392588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:14.392588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:14.392588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:14.392588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:14.392588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:14.392588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:14.392588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:14.392588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:14.392588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:14.392588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:14.392588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:14.392588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:14.392588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:14.392588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:14.393542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:14.396189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:14.396189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:14.396189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:14.396189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 289/598] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:14.396653 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fb944-b325-49f6-85f1-9bd400ec3a30 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:14.464168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 310/599] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:54:14 2022] PUT /identity/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/a899c6dabca948269f6c1370425f244b/roles/9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:54:14.465941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52196254-3a93-4671-b1ec-a04cf6c39e65 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:14.466225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52196254-3a93-4671-b1ec-a04cf6c39e65 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:14.466494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52196254-3a93-4671-b1ec-a04cf6c39e65 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:14.468072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 311/600] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:14.474316 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52196254-3a93-4671-b1ec-a04cf6c39e65 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:14.537526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fb944-b325-49f6-85f1-9bd400ec3a30 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:14.557577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:14.581152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6aa60-e3f8-44c3-9d29-f5d10a97c422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:14.581828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6aa60-e3f8-44c3-9d29-f5d10a97c422 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:14.582307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6aa60-e3f8-44c3-9d29-f5d10a97c422 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:14.583085 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6aa60-e3f8-44c3-9d29-f5d10a97c422 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:14.584094 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6aa60-e3f8-44c3-9d29-f5d10a97c422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:14.584706 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6aa60-e3f8-44c3-9d29-f5d10a97c422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:14.587253 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6aa60-e3f8-44c3-9d29-f5d10a97c422 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:14.594533 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:14.606076 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 290/601] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:14.606076 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b25ae4-b109-41e1-872f-ba0911f89d57 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:14.606076 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b25ae4-b109-41e1-872f-ba0911f89d57 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:14.606076 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b25ae4-b109-41e1-872f-ba0911f89d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:14.615796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52196254-3a93-4671-b1ec-a04cf6c39e65 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:14.625112 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-46b25ae4-b109-41e1-872f-ba0911f89d57 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:14.627384 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-46b25ae4-b109-41e1-872f-ba0911f89d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:14.662304 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c2433-d53d-450a-b7f0-b5cca61df292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:14.663226 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c2433-d53d-450a-b7f0-b5cca61df292 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:14.663589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c2433-d53d-450a-b7f0-b5cca61df292 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:14.663938 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c2433-d53d-450a-b7f0-b5cca61df292 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:14.664673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c2433-d53d-450a-b7f0-b5cca61df292 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:14.665203 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c2433-d53d-450a-b7f0-b5cca61df292 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:14.666902 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c2433-d53d-450a-b7f0-b5cca61df292 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:14.682545 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 312/602] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:14.683819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c2433-d53d-450a-b7f0-b5cca61df292 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:14.713229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 291/603] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:14.714298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b25ae4-b109-41e1-872f-ba0911f89d57 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:14.822875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c2433-d53d-450a-b7f0-b5cca61df292 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:14.861052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b25ae4-b109-41e1-872f-ba0911f89d57 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:14.863366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:14.865773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f4f5a-17d9-469e-b9db-d95d7785a235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:14.866727 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f4f5a-17d9-469e-b9db-d95d7785a235 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:14.867213 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f4f5a-17d9-469e-b9db-d95d7785a235 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:14.867677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f4f5a-17d9-469e-b9db-d95d7785a235 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:14.875369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f4f5a-17d9-469e-b9db-d95d7785a235 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:14.877240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f4f5a-17d9-469e-b9db-d95d7785a235 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:14.890865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1c5f4f5a-17d9-469e-b9db-d95d7785a235 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:14.890865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:14.890865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:14.890865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:14.890865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:14.890865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:14.890865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:14.890865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:14.890865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:14.890865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:14.890865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:14.890865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:14.890865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:14.890865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:14.890865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:14.893184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:14.894263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:14.894263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:14.894263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:14.894263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:14.894263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:14.894263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:14.894263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:14.894263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:14.894263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 313/604] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:14.900863 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1f5f7-caf4-4f16-a651-87bd17595bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:14.901256 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f4f5a-17d9-469e-b9db-d95d7785a235 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:14.901500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f5f7-caf4-4f16-a651-87bd17595bbb None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:14.901777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f5f7-caf4-4f16-a651-87bd17595bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:14.902046 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f5f7-caf4-4f16-a651-87bd17595bbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:14.909510 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1f5f7-caf4-4f16-a651-87bd17595bbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:14.911055 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1f5f7-caf4-4f16-a651-87bd17595bbb None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:14.925020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:14.931500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-ddd1f5f7-caf4-4f16-a651-87bd17595bbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:14.931500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:14.931500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:14.931500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:14.931500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:14.931500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:14.931500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:14.931500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:14.931500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:14.931500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:14.931500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:14.931500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:14.931500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:14.931500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:14.931500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:14.932420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:14.933385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:14.933385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:14.933385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:14.933385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:14.933385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:14.933385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:14.933385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:14.933385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:14.935814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 292/605] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:14.941741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1f5f7-caf4-4f16-a651-87bd17595bbb None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:15.098217 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1f5f7-caf4-4f16-a651-87bd17595bbb None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:15.135271 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f4f5a-17d9-469e-b9db-d95d7785a235 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:15.148462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2120222-cb65-47e5-8ac4-5ef540b303ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:15.149215 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2120222-cb65-47e5-8ac4-5ef540b303ec None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:15.149586 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2120222-cb65-47e5-8ac4-5ef540b303ec None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:15.149971 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2120222-cb65-47e5-8ac4-5ef540b303ec None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:15.150763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2120222-cb65-47e5-8ac4-5ef540b303ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:15.151190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2120222-cb65-47e5-8ac4-5ef540b303ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:15.153051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2120222-cb65-47e5-8ac4-5ef540b303ec None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:15.174393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 293/606] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:15.174758 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b10e2-14a7-4a7a-8744-207a7f305fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:15.175607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b10e2-14a7-4a7a-8744-207a7f305fac None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:15.176048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b10e2-14a7-4a7a-8744-207a7f305fac None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:15.176474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b10e2-14a7-4a7a-8744-207a7f305fac None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:15.177326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b10e2-14a7-4a7a-8744-207a7f305fac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:15.177772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b10e2-14a7-4a7a-8744-207a7f305fac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:15.179962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b10e2-14a7-4a7a-8744-207a7f305fac None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:15.180192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2120222-cb65-47e5-8ac4-5ef540b303ec None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:15.196337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 314/607] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:15.197426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b10e2-14a7-4a7a-8744-207a7f305fac None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:15.219989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:15.272402 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:15.308307 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b10e2-14a7-4a7a-8744-207a7f305fac None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:15.327944 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2120222-cb65-47e5-8ac4-5ef540b303ec None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:15.352623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44011ec-6f1f-4177-87fc-0e6d83036110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:15.353543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44011ec-6f1f-4177-87fc-0e6d83036110 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:15.353971 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44011ec-6f1f-4177-87fc-0e6d83036110 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:15.354446 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44011ec-6f1f-4177-87fc-0e6d83036110 None admin] PATH_INFO: `/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/30909d152a5c4a1fac6c1a2765868177/roles/9f0e36c2ed8843fbabd3a4cf30bc3725` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:15.363422 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a4436-1122-47a1-9241-ae647ee9689c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:15.364050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a4436-1122-47a1-9241-ae647ee9689c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:15.364323 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a4436-1122-47a1-9241-ae647ee9689c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:15.364593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a4436-1122-47a1-9241-ae647ee9689c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:15.372762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44011ec-6f1f-4177-87fc-0e6d83036110 None admin] RBAC: Authorizing `identity:create_grant(project_id=399f0095cc8b4c64893ccc75af13c52d, user_id=30909d152a5c4a1fac6c1a2765868177, role_id=9f0e36c2ed8843fbabd3a4cf30bc3725)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:15.374783 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44011ec-6f1f-4177-87fc-0e6d83036110 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:15.376077 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a4436-1122-47a1-9241-ae647ee9689c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:15.383003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-275a4436-1122-47a1-9241-ae647ee9689c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:15.383003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:15.383003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:15.383003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:15.383003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:15.383003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:15.383003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:15.383003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:15.383003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:15.383003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:15.383003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:15.383003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:15.383003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:15.383003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:15.383003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:15.383003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:15.384243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:15.384243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:15.384243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:15.384243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:15.384243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:15.384243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:15.384243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:15.384243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:15.384243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:15.384243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:15.384243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:15.384243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:15.384243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:15.384243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:15.384243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:15.384243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:15.385200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:15.385200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:15.385200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:15.385200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:15.385200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:15.385200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:15.385200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:15.385200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:15.385200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:15.385200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:15.385200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:15.385200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:15.387076 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 294/608] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:15.388062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a4436-1122-47a1-9241-ae647ee9689c None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:15.491409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 315/609] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:54:15 2022] PUT /identity/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/30909d152a5c4a1fac6c1a2765868177/roles/9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:54:15.491652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:15.491748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44011ec-6f1f-4177-87fc-0e6d83036110 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:15.520453 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a4436-1122-47a1-9241-ae647ee9689c None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:15.594220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9068cf6-ff68-4b70-ba03-c584abbdabbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:15.595653 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9068cf6-ff68-4b70-ba03-c584abbdabbb None admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:15.596087 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9068cf6-ff68-4b70-ba03-c584abbdabbb None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:15.596501 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9068cf6-ff68-4b70-ba03-c584abbdabbb None admin] PATH_INFO: `/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/bf6798f89acf44e897c481924edd9047/roles/9f0e36c2ed8843fbabd3a4cf30bc3725` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:15.626130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44011ec-6f1f-4177-87fc-0e6d83036110 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:15.633381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9068cf6-ff68-4b70-ba03-c584abbdabbb None admin] RBAC: Authorizing `identity:create_grant(project_id=399f0095cc8b4c64893ccc75af13c52d, user_id=bf6798f89acf44e897c481924edd9047, role_id=9f0e36c2ed8843fbabd3a4cf30bc3725)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:15.635377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9068cf6-ff68-4b70-ba03-c584abbdabbb None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:15.648082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:15.668182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb9bf2-9ac1-4418-a335-90894ea66234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:15.668863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb9bf2-9ac1-4418-a335-90894ea66234 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:15.669199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb9bf2-9ac1-4418-a335-90894ea66234 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:15.669467 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb9bf2-9ac1-4418-a335-90894ea66234 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:15.670190 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb9bf2-9ac1-4418-a335-90894ea66234 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:15.670496 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb9bf2-9ac1-4418-a335-90894ea66234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:15.672037 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb9bf2-9ac1-4418-a335-90894ea66234 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:15.687047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 316/610] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:15.688353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb9bf2-9ac1-4418-a335-90894ea66234 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:15.744409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 295/611] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:54:15 2022] PUT /identity/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/bf6798f89acf44e897c481924edd9047/roles/9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:54:15.817004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb9bf2-9ac1-4418-a335-90894ea66234 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:15.855886 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955d794-17c0-4766-9987-cf66c7ed7e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:15.856805 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955d794-17c0-4766-9987-cf66c7ed7e17 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:15.857230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955d794-17c0-4766-9987-cf66c7ed7e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:15.857646 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955d794-17c0-4766-9987-cf66c7ed7e17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:15.869902 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955d794-17c0-4766-9987-cf66c7ed7e17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:15.871742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955d794-17c0-4766-9987-cf66c7ed7e17 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:15.888877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3955d794-17c0-4766-9987-cf66c7ed7e17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:15.888877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:15.888877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:15.888877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:15.888877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:15.888877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:15.888877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:15.888877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:15.888877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:15.888877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:15.888877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:15.888877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:15.888877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:15.888877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:15.888877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:15.888877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:15.890102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:15.891222 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:15.891222 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:15.891222 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:15.895067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 317/612] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:15.899712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9068cf6-ff68-4b70-ba03-c584abbdabbb None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:16.033543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6929e72-3512-45b2-972a-b0ea84130115 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:16.033947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6929e72-3512-45b2-972a-b0ea84130115 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:16.034127 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6929e72-3512-45b2-972a-b0ea84130115 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:16.039808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 318/613] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:16.046610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd4908-8748-48dd-a014-b138d27e47a4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:16.046893 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd4908-8748-48dd-a014-b138d27e47a4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:16.047186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd4908-8748-48dd-a014-b138d27e47a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:16.065699 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9068cf6-ff68-4b70-ba03-c584abbdabbb None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:16.073028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-edcd4908-8748-48dd-a014-b138d27e47a4 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:16.074529 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-edcd4908-8748-48dd-a014-b138d27e47a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:16.113750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:16.122557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6b210-1466-47e8-9c38-caafab819434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:16.123194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6b210-1466-47e8-9c38-caafab819434 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:16.123725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6b210-1466-47e8-9c38-caafab819434 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:16.124001 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6b210-1466-47e8-9c38-caafab819434 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:16.124652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e6b210-1466-47e8-9c38-caafab819434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:16.124952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e6b210-1466-47e8-9c38-caafab819434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:16.126333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e6b210-1466-47e8-9c38-caafab819434 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:16.128170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:16.139186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 296/614] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:16.148550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6b210-1466-47e8-9c38-caafab819434 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:16.186325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 319/615] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:16.327235 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6b210-1466-47e8-9c38-caafab819434 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:16.362336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040c4c2-f87c-477b-9a75-b89248ee6312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:16.362988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040c4c2-f87c-477b-9a75-b89248ee6312 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:16.363268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040c4c2-f87c-477b-9a75-b89248ee6312 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:16.363534 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040c4c2-f87c-477b-9a75-b89248ee6312 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:16.371745 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040c4c2-f87c-477b-9a75-b89248ee6312 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:16.373212 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040c4c2-f87c-477b-9a75-b89248ee6312 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:16.385092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-b040c4c2-f87c-477b-9a75-b89248ee6312 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 07:54:16.385092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:16.385092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:16.385092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:16.385092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:16.385092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:16.385092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:16.385092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:16.385092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:16.385092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:16.385092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:16.385092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:16.385092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:16.385092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:16.385092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:16.385092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:16.386082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:16.387064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:16.387064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:16.387064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:16.387064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:16.387064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:16.387064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:16.387064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:16.387064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:16.387064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 07:54:16.387064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:16.390872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 297/616] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:54:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:16.392288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd4908-8748-48dd-a014-b138d27e47a4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:16.544020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd4908-8748-48dd-a014-b138d27e47a4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:16.589458 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef540d9-6cc5-49af-a0b3-1c24148cbe85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:16.590258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef540d9-6cc5-49af-a0b3-1c24148cbe85 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:16.590634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef540d9-6cc5-49af-a0b3-1c24148cbe85 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:16.591038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef540d9-6cc5-49af-a0b3-1c24148cbe85 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:16.591908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef540d9-6cc5-49af-a0b3-1c24148cbe85 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:16.592344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef540d9-6cc5-49af-a0b3-1c24148cbe85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:16.593946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef540d9-6cc5-49af-a0b3-1c24148cbe85 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:16.606297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040c4c2-f87c-477b-9a75-b89248ee6312 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:16.629972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 320/617] 10.222.0.30 () {56 vars in 1135 bytes} [Thu Oct 6 07:54:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:16.633065 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef540d9-6cc5-49af-a0b3-1c24148cbe85 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:16.695338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:16.735564 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:16.802628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef540d9-6cc5-49af-a0b3-1c24148cbe85 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:16.831141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040c4c2-f87c-477b-9a75-b89248ee6312 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:16.864753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79a0bd-5ddb-4de5-845f-0fa2c6a5a68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:16.865685 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79a0bd-5ddb-4de5-845f-0fa2c6a5a68a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:16.867544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79a0bd-5ddb-4de5-845f-0fa2c6a5a68a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:16.867979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79a0bd-5ddb-4de5-845f-0fa2c6a5a68a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:16.888500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79a0bd-5ddb-4de5-845f-0fa2c6a5a68a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:16.891457 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79a0bd-5ddb-4de5-845f-0fa2c6a5a68a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:16.919898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-7f79a0bd-5ddb-4de5-845f-0fa2c6a5a68a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:16.919898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:16.919898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:16.919898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:16.919898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:16.919898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:16.919898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:16.919898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:16.919898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:16.919898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:16.919898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:16.919898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:16.919898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:16.919898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:16.919898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:16.919898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:16.921261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:16.922338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:16.922338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:16.922338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:16.923970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81bcf5-66ac-4d9c-9165-86806ff1f9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:16.924764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81bcf5-66ac-4d9c-9165-86806ff1f9ed None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:16.925178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81bcf5-66ac-4d9c-9165-86806ff1f9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:16.925604 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81bcf5-66ac-4d9c-9165-86806ff1f9ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:16.935257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 321/618] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:16.942307 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79a0bd-5ddb-4de5-845f-0fa2c6a5a68a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:16.945605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c81bcf5-66ac-4d9c-9165-86806ff1f9ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:16.957314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-1c81bcf5-66ac-4d9c-9165-86806ff1f9ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:16.957314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:16.957314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:16.957314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:16.957314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:16.957314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:16.957314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:16.957314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:16.957314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:16.957314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:16.957314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:16.957314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:16.957314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:16.957314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:16.957314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:16.957314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:16.958305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:16.958305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:16.958305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:16.958305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:16.958305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:16.958305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:16.958305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:16.958305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:16.958305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:16.958305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:16.958305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:16.958305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:16.958305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:16.958305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:16.958305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:16.958305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:16.959168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:16.959168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:16.959168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:16.959168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:16.959168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:16.959168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:16.959168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:16.959168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:16.959168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:16.959168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:16.959168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:16.959168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:16.965213 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 298/619] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:16.981977 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81bcf5-66ac-4d9c-9165-86806ff1f9ed None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:17.091416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:17.096700 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79a0bd-5ddb-4de5-845f-0fa2c6a5a68a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:17.163970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81bcf5-66ac-4d9c-9165-86806ff1f9ed None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:17.164999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b31fc1-6d31-4127-9f05-b75ffb06d47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:17.165615 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b31fc1-6d31-4127-9f05-b75ffb06d47c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:17.165889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b31fc1-6d31-4127-9f05-b75ffb06d47c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:17.166165 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b31fc1-6d31-4127-9f05-b75ffb06d47c None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:17.166818 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b31fc1-6d31-4127-9f05-b75ffb06d47c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:17.167147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b31fc1-6d31-4127-9f05-b75ffb06d47c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:17.169865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b31fc1-6d31-4127-9f05-b75ffb06d47c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:17.189649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 322/620] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:17.189977 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:17.191023 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc29d-ea6c-4722-a8e6-0ecbc013f1b4 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:17.191234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc29d-ea6c-4722-a8e6-0ecbc013f1b4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:17.191516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc29d-ea6c-4722-a8e6-0ecbc013f1b4 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:17.192978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 323/621] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:17.198101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc29d-ea6c-4722-a8e6-0ecbc013f1b4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:17.220279 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0aa2a-f2a7-45e7-97e4-b7542205c10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:17.221397 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0aa2a-f2a7-45e7-97e4-b7542205c10d None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:17.222143 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0aa2a-f2a7-45e7-97e4-b7542205c10d None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:17.222707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0aa2a-f2a7-45e7-97e4-b7542205c10d None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:17.223891 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0aa2a-f2a7-45e7-97e4-b7542205c10d None admin] RBAC: Adding query filter params (name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:17.224653 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0aa2a-f2a7-45e7-97e4-b7542205c10d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:17.228127 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0aa2a-f2a7-45e7-97e4-b7542205c10d None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:17.243108 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 299/622] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:17.243108 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8d14e-cd32-400c-b017-5927ab60af0b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:17.243108 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8d14e-cd32-400c-b017-5927ab60af0b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:17.243108 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8d14e-cd32-400c-b017-5927ab60af0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:17.269287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-abd8d14e-cd32-400c-b017-5927ab60af0b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:17.270665 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-abd8d14e-cd32-400c-b017-5927ab60af0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:17.322514 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc29d-ea6c-4722-a8e6-0ecbc013f1b4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:17.341164 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 300/623] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:17.342276 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8d14e-cd32-400c-b017-5927ab60af0b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:17.354835 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:17.361363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9657eb7-5d45-4bf9-8726-3a99859bdc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:17.362270 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9657eb7-5d45-4bf9-8726-3a99859bdc33 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:17.362689 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9657eb7-5d45-4bf9-8726-3a99859bdc33 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:17.363134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9657eb7-5d45-4bf9-8726-3a99859bdc33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:17.372014 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9657eb7-5d45-4bf9-8726-3a99859bdc33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:17.374019 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9657eb7-5d45-4bf9-8726-3a99859bdc33 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:17.394970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d9657eb7-5d45-4bf9-8726-3a99859bdc33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:17.394970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:17.394970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:17.394970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:17.394970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:17.394970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:17.394970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:17.394970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:17.394970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:17.394970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:17.394970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:17.394970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:17.394970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:17.394970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:17.394970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:17.395929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:17.397104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:17.397104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:17.397104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:17.397104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:17.397104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:17.397104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:17.397104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:17.397104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:17.397104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 324/624] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:17.408128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9657eb7-5d45-4bf9-8726-3a99859bdc33 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:17.525180 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8d14e-cd32-400c-b017-5927ab60af0b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:17.537199 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:17.566738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9657eb7-5d45-4bf9-8726-3a99859bdc33 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:17.605559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635a4ca-2198-4828-8abb-46e54b4717c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:17.606084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635a4ca-2198-4828-8abb-46e54b4717c5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:17.606358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635a4ca-2198-4828-8abb-46e54b4717c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:17.606628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635a4ca-2198-4828-8abb-46e54b4717c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:17.607358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c635a4ca-2198-4828-8abb-46e54b4717c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:17.608119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c635a4ca-2198-4828-8abb-46e54b4717c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:17.609710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c635a4ca-2198-4828-8abb-46e54b4717c5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:17.624458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401c318-2233-4e43-b22d-30d37e6e8f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:17.625154 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 325/625] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:17.625260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401c318-2233-4e43-b22d-30d37e6e8f3a None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:17.625418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401c318-2233-4e43-b22d-30d37e6e8f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:17.625650 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401c318-2233-4e43-b22d-30d37e6e8f3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:17.630813 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635a4ca-2198-4828-8abb-46e54b4717c5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:17.646671 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401c318-2233-4e43-b22d-30d37e6e8f3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:17.648554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401c318-2233-4e43-b22d-30d37e6e8f3a None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:17.681404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-3401c318-2233-4e43-b22d-30d37e6e8f3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:17.681404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:17.681404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:17.681404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:17.681404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:17.681404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:17.681404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:17.681404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:17.681404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:17.681404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:17.681404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:17.681404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:17.681404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:17.681404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:17.681404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:17.681404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:17.682401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:17.683327 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:17.683327 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:17.683327 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:17.688921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:17.689658 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 301/626] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:17.690572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401c318-2233-4e43-b22d-30d37e6e8f3a None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:17.746315 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635a4ca-2198-4828-8abb-46e54b4717c5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:17.789506 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889f79c-1711-42a2-8dfc-b596dc6d240e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:17.790402 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889f79c-1711-42a2-8dfc-b596dc6d240e None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:17.790852 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889f79c-1711-42a2-8dfc-b596dc6d240e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:17.791530 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889f79c-1711-42a2-8dfc-b596dc6d240e None admin] PATH_INFO: `/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/3fa1c430c9f7416eaf1d17300b827e05/roles/9f0e36c2ed8843fbabd3a4cf30bc3725` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:17.797839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401c318-2233-4e43-b22d-30d37e6e8f3a None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:17.812262 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889f79c-1711-42a2-8dfc-b596dc6d240e None admin] RBAC: Authorizing `identity:create_grant(project_id=399f0095cc8b4c64893ccc75af13c52d, user_id=3fa1c430c9f7416eaf1d17300b827e05, role_id=9f0e36c2ed8843fbabd3a4cf30bc3725)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:17.814098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889f79c-1711-42a2-8dfc-b596dc6d240e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:17.833970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32774922-02b2-41c2-ad47-40657300a162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:17.834702 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32774922-02b2-41c2-ad47-40657300a162 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:17.835088 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32774922-02b2-41c2-ad47-40657300a162 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:17.835427 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32774922-02b2-41c2-ad47-40657300a162 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:17.845745 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32774922-02b2-41c2-ad47-40657300a162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:17.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-32774922-02b2-41c2-ad47-40657300a162 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:17.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:17.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:17.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:17.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:17.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:17.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:17.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:17.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:17.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:17.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:17.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:17.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:17.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:17.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:17.851952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:17.852902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:17.852902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:17.852902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:17.852902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:17.852902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:17.852902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:17.852902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:17.852902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:17.852902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:17.852902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:17.852902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:17.852902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:17.852902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:17.852902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:17.852902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:17.852902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:17.853839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:17.853839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:17.853839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:17.853839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:17.853839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:17.853839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:17.853839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:17.853839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:17.853839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:17.853839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:17.853839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:17.853839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:17.859677 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 302/627] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:17.859677 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32774922-02b2-41c2-ad47-40657300a162 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:17.882220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:17.901186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 326/628] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:54:17 2022] PUT /identity/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/3fa1c430c9f7416eaf1d17300b827e05/roles/9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 07:54:17.902193 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889f79c-1711-42a2-8dfc-b596dc6d240e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:18.033869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32774922-02b2-41c2-ad47-40657300a162 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:18.066701 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889f79c-1711-42a2-8dfc-b596dc6d240e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:18.068945 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:18.087409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a7c1f-f58b-40c4-8aa6-455dd32de57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:18.088185 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a7c1f-f58b-40c4-8aa6-455dd32de57c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:18.088840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a7c1f-f58b-40c4-8aa6-455dd32de57c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:18.089122 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a7c1f-f58b-40c4-8aa6-455dd32de57c None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:18.089843 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a7c1f-f58b-40c4-8aa6-455dd32de57c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:18.090148 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a7c1f-f58b-40c4-8aa6-455dd32de57c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:18.091646 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a7c1f-f58b-40c4-8aa6-455dd32de57c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:18.106653 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 303/629] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:18.107911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36af1b0-93a3-4a0f-aa34-1d2934b50d0d None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:18.108202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36af1b0-93a3-4a0f-aa34-1d2934b50d0d None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:18.108471 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36af1b0-93a3-4a0f-aa34-1d2934b50d0d None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:18.109995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 304/630] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:18.110119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08ab57-ce01-481e-afcb-7b5f07e70895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:18.110608 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08ab57-ce01-481e-afcb-7b5f07e70895 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:18.110882 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08ab57-ce01-481e-afcb-7b5f07e70895 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:18.111192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08ab57-ce01-481e-afcb-7b5f07e70895 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:18.112250 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08ab57-ce01-481e-afcb-7b5f07e70895 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:18.112597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08ab57-ce01-481e-afcb-7b5f07e70895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:18.114052 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08ab57-ce01-481e-afcb-7b5f07e70895 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:18.114605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36af1b0-93a3-4a0f-aa34-1d2934b50d0d None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:18.131015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 327/631] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:18.131015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ace7df-6d05-449e-a6c2-c4a54b022ca9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:18.131015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ace7df-6d05-449e-a6c2-c4a54b022ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:18.131015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ace7df-6d05-449e-a6c2-c4a54b022ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:18.162986 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f6ace7df-6d05-449e-a6c2-c4a54b022ca9 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:18.162986 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f6ace7df-6d05-449e-a6c2-c4a54b022ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:18.223344 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:18.235906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36af1b0-93a3-4a0f-aa34-1d2934b50d0d None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:18.243020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 328/632] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:18.244010 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ace7df-6d05-449e-a6c2-c4a54b022ca9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:18.272012 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473310f-dbfa-4f27-aa92-3673479eee81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:18.272648 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473310f-dbfa-4f27-aa92-3673479eee81 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:18.272927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473310f-dbfa-4f27-aa92-3673479eee81 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:18.273200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473310f-dbfa-4f27-aa92-3673479eee81 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:18.280612 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473310f-dbfa-4f27-aa92-3673479eee81 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:18.282065 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473310f-dbfa-4f27-aa92-3673479eee81 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:18.294569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-b473310f-dbfa-4f27-aa92-3673479eee81 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:54:18.294569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:18.294569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:18.294569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:18.294569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:18.294569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:18.294569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:18.294569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:18.294569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:18.294569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:18.294569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:18.294569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:18.294569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:18.294569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:18.294569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:18.294569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:18.295616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:18.296972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:18.296972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:18.296972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:18.296972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:18.296972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:18.296972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:18.296972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:18.296972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:18.296972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:54:18.296972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:18.296972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 305/633] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Oct 6 07:54:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:18.301835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473310f-dbfa-4f27-aa92-3673479eee81 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:18.357552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:18.364902 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ace7df-6d05-449e-a6c2-c4a54b022ca9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:18.415324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ab940-d8ae-4c04-8f28-7c3b2131c541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:18.416554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ab940-d8ae-4c04-8f28-7c3b2131c541 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:18.416834 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ab940-d8ae-4c04-8f28-7c3b2131c541 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:18.417107 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ab940-d8ae-4c04-8f28-7c3b2131c541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:18.436003 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ab940-d8ae-4c04-8f28-7c3b2131c541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:18.437459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ab940-d8ae-4c04-8f28-7c3b2131c541 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:18.455224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-252ab940-d8ae-4c04-8f28-7c3b2131c541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:18.455224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:18.455224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:18.455224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:18.455224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:18.455224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:18.455224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:18.455224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:18.455224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:18.455224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:18.455224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:18.455224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:18.455224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:18.455224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:18.455224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:18.455224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:18.456978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:18.458158 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:18.458158 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:18.458158 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:18.458158 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 329/634] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:18.459014 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb850f-6896-4fe1-8730-661973e01a08 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:18.459014 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb850f-6896-4fe1-8730-661973e01a08 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:18.459220 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb850f-6896-4fe1-8730-661973e01a08 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:18.461206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 330/635] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:18.472215 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd2df7-868e-41e5-86ae-f6ea748e7011 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:18.472494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd2df7-868e-41e5-86ae-f6ea748e7011 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:18.475495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd2df7-868e-41e5-86ae-f6ea748e7011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:18.488565 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473310f-dbfa-4f27-aa92-3673479eee81 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:18.494597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-57bd2df7-868e-41e5-86ae-f6ea748e7011 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:18.495831 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-57bd2df7-868e-41e5-86ae-f6ea748e7011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:18.526607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a721f-3c75-432e-8d23-474f94d3ef1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:18.527444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a721f-3c75-432e-8d23-474f94d3ef1c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:18.528043 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a721f-3c75-432e-8d23-474f94d3ef1c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:18.528332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a721f-3c75-432e-8d23-474f94d3ef1c None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:18.529354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a721f-3c75-432e-8d23-474f94d3ef1c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:18.529752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a721f-3c75-432e-8d23-474f94d3ef1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:18.531535 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a721f-3c75-432e-8d23-474f94d3ef1c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:18.554448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:18.558853 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 306/636] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Oct 6 07:54:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:18.559755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a721f-3c75-432e-8d23-474f94d3ef1c None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:18.574605 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 331/637] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:18.575680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd2df7-868e-41e5-86ae-f6ea748e7011 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:18.655903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:18.673622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a721f-3c75-432e-8d23-474f94d3ef1c None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:18.681745 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd2df7-868e-41e5-86ae-f6ea748e7011 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:18.711017 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab781a2-3f0c-411a-97ea-0eba22592d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:18.711623 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab781a2-3f0c-411a-97ea-0eba22592d87 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:18.711901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab781a2-3f0c-411a-97ea-0eba22592d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:18.712172 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab781a2-3f0c-411a-97ea-0eba22592d87 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:18.712829 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab781a2-3f0c-411a-97ea-0eba22592d87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:18.713121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab781a2-3f0c-411a-97ea-0eba22592d87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:18.714507 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab781a2-3f0c-411a-97ea-0eba22592d87 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:18.720674 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef748da8-134e-4716-a9a2-613727800387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:18.721567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef748da8-134e-4716-a9a2-613727800387 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:18.722048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef748da8-134e-4716-a9a2-613727800387 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:18.722510 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef748da8-134e-4716-a9a2-613727800387 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:18.727687 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 307/638] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:18.728532 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab781a2-3f0c-411a-97ea-0eba22592d87 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:18.793963 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:18.797590 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef748da8-134e-4716-a9a2-613727800387 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:18.799181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef748da8-134e-4716-a9a2-613727800387 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:18.840495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab781a2-3f0c-411a-97ea-0eba22592d87 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:18.875570 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8077f9a-d43d-46ef-a106-dd19301a209c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:18.876521 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8077f9a-d43d-46ef-a106-dd19301a209c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:18.877003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8077f9a-d43d-46ef-a106-dd19301a209c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:18.877456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8077f9a-d43d-46ef-a106-dd19301a209c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:18.884027 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:18.889577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8077f9a-d43d-46ef-a106-dd19301a209c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:18.894973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-c8077f9a-d43d-46ef-a106-dd19301a209c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:18.894973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:18.894973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:18.894973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:18.894973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:18.894973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:18.894973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:18.894973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:18.894973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:18.894973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:18.894973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:18.894973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:18.894973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:18.894973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:18.894973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:18.894973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:18.899356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:18.899356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:18.899356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:18.899356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:18.899356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:18.899356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:18.899356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:18.899356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:18.899356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:18.899356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:18.899356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:18.899356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:18.899356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:18.899356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:18.899356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:18.899356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:18.901533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:18.901533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:18.901533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:18.901533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:18.901533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:18.901533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:18.901533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:18.901533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:18.901533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:18.901533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:18.901533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:18.901533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:18.901533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 308/639] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:18.901533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8077f9a-d43d-46ef-a106-dd19301a209c None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:18.917540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 332/640] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:54:18.918596 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef748da8-134e-4716-a9a2-613727800387 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:19.007666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8077f9a-d43d-46ef-a106-dd19301a209c None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:19.024473 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef748da8-134e-4716-a9a2-613727800387 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:19.025708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:19.043876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b83d5a-90fb-437d-8447-eaa421c610ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:19.044762 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b83d5a-90fb-437d-8447-eaa421c610ba None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:19.045218 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b83d5a-90fb-437d-8447-eaa421c610ba None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:19.045666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b83d5a-90fb-437d-8447-eaa421c610ba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:19.053370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b83d5a-90fb-437d-8447-eaa421c610ba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:19.054833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b83d5a-90fb-437d-8447-eaa421c610ba None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:19.061465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8949215-9a9c-47ef-b606-94589cab410c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:19.062059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8949215-9a9c-47ef-b606-94589cab410c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:19.062332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8949215-9a9c-47ef-b606-94589cab410c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:19.062604 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8949215-9a9c-47ef-b606-94589cab410c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:19.066458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-25b83d5a-90fb-437d-8447-eaa421c610ba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 07:54:19.066458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:19.066458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:19.066458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:19.066458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:19.066458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:19.066458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:19.066458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:19.066458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:19.066458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:19.066458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:19.066458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:19.066458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:19.066458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:19.066458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:19.066458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:19.067463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:19.068448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:19.068448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:19.068448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:19.068448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:19.068448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:19.068448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:19.068448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:19.068448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:19.068448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 07:54:19.068448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:19.071888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 309/641] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:54:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:19.072672 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b83d5a-90fb-437d-8447-eaa421c610ba None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:19.081170 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8949215-9a9c-47ef-b606-94589cab410c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:19.087547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d8949215-9a9c-47ef-b606-94589cab410c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:19.087547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:19.087547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:19.087547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:19.087547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:19.087547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:19.087547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:19.087547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:19.087547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:19.087547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:19.087547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:19.087547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:19.087547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:19.087547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:19.087547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:19.087547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:19.088548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:19.088548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:19.088548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:19.088548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:19.088548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:19.088548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:19.088548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:19.088548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:19.088548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:19.088548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:19.088548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:19.088548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:19.088548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:19.088548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:19.088548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:19.088548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:19.089559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:19.089559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:19.089559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:19.089559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:19.089559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:19.089559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:19.089559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:19.089559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:19.089559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:19.089559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:19.089559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:19.089559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:19.090640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 333/642] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:19.091768 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8949215-9a9c-47ef-b606-94589cab410c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:19.132167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:19.189660 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b83d5a-90fb-437d-8447-eaa421c610ba None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:19.207451 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8949215-9a9c-47ef-b606-94589cab410c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:19.228569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52e7d5-2268-4575-9248-f2fbd9d1161d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:19.229185 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52e7d5-2268-4575-9248-f2fbd9d1161d None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:19.229456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52e7d5-2268-4575-9248-f2fbd9d1161d None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:19.229723 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52e7d5-2268-4575-9248-f2fbd9d1161d None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:19.230377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52e7d5-2268-4575-9248-f2fbd9d1161d None admin] RBAC: Adding query filter params (name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:19.230674 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52e7d5-2268-4575-9248-f2fbd9d1161d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:19.232085 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52e7d5-2268-4575-9248-f2fbd9d1161d None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:19.243203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 310/643] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:19.244078 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52e7d5-2268-4575-9248-f2fbd9d1161d None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:19.245080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8007b-3490-4e4f-8902-3ca24477985b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:19.245714 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8007b-3490-4e4f-8902-3ca24477985b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:19.245990 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8007b-3490-4e4f-8902-3ca24477985b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:19.246256 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8007b-3490-4e4f-8902-3ca24477985b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:19.261703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8007b-3490-4e4f-8902-3ca24477985b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:19.263462 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8007b-3490-4e4f-8902-3ca24477985b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:19.281352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-78a8007b-3490-4e4f-8902-3ca24477985b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:19.281352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:19.281352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:19.281352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:19.281352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:19.281352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:19.281352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:19.281352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:19.281352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:19.281352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:19.281352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:19.281352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:19.281352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:19.281352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:19.281352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:19.281352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:19.282318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:19.283418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:19.283418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:19.283418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:19.283418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 334/644] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:19.283418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8007b-3490-4e4f-8902-3ca24477985b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:19.292107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:19.354874 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52e7d5-2268-4575-9248-f2fbd9d1161d None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:19.384489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:19.388826 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895978f1-a3b1-4e86-a197-142d59a2095e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:19.389424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895978f1-a3b1-4e86-a197-142d59a2095e None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:19.389699 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895978f1-a3b1-4e86-a197-142d59a2095e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:19.390025 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895978f1-a3b1-4e86-a197-142d59a2095e None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:19.390718 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895978f1-a3b1-4e86-a197-142d59a2095e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:19.391288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895978f1-a3b1-4e86-a197-142d59a2095e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:19.393398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895978f1-a3b1-4e86-a197-142d59a2095e None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:19.396570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8007b-3490-4e4f-8902-3ca24477985b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:19.415367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 311/645] 10.222.0.30 () {56 vars in 1137 bytes} [Thu Oct 6 07:54:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:19.415367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895978f1-a3b1-4e86-a197-142d59a2095e None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:19.441793 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cc5a4-5244-4df7-8cc8-e70a3e83ce83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:19.442317 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cc5a4-5244-4df7-8cc8-e70a3e83ce83 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:19.442589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cc5a4-5244-4df7-8cc8-e70a3e83ce83 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:19.442856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cc5a4-5244-4df7-8cc8-e70a3e83ce83 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:19.443538 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345cc5a4-5244-4df7-8cc8-e70a3e83ce83 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:19.443949 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345cc5a4-5244-4df7-8cc8-e70a3e83ce83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:19.445271 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345cc5a4-5244-4df7-8cc8-e70a3e83ce83 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:19.458196 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 335/646] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:19.458196 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cc5a4-5244-4df7-8cc8-e70a3e83ce83 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:19.515370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:19.522576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895978f1-a3b1-4e86-a197-142d59a2095e None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:19.560104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12495615-b66f-4e5b-867b-58dc2b0ce100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:19.561054 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12495615-b66f-4e5b-867b-58dc2b0ce100 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:19.561479 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12495615-b66f-4e5b-867b-58dc2b0ce100 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:19.561879 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12495615-b66f-4e5b-867b-58dc2b0ce100 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:19.566136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cc5a4-5244-4df7-8cc8-e70a3e83ce83 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:19.574152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12495615-b66f-4e5b-867b-58dc2b0ce100 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:19.576069 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12495615-b66f-4e5b-867b-58dc2b0ce100 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:19.596726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-12495615-b66f-4e5b-867b-58dc2b0ce100 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:19.596726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:19.596726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:19.596726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:19.596726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:19.596726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:19.596726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:19.596726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:19.596726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:19.596726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:19.596726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:19.596726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:19.596726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:19.596726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:19.596726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:19.596726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:19.597698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:19.598592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:19.598592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:19.598592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:19.598592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 312/647] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:19.598592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12495615-b66f-4e5b-867b-58dc2b0ce100 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:19.605510 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44db28-aa2e-4ebb-8ca7-3a69dcc0aa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:19.606127 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44db28-aa2e-4ebb-8ca7-3a69dcc0aa3e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:19.606400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44db28-aa2e-4ebb-8ca7-3a69dcc0aa3e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:19.606668 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44db28-aa2e-4ebb-8ca7-3a69dcc0aa3e None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:19.607338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44db28-aa2e-4ebb-8ca7-3a69dcc0aa3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:19.607638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44db28-aa2e-4ebb-8ca7-3a69dcc0aa3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:19.609485 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44db28-aa2e-4ebb-8ca7-3a69dcc0aa3e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:19.627192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 336/648] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:19.627192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44db28-aa2e-4ebb-8ca7-3a69dcc0aa3e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:19.635774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:19.712734 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12495615-b66f-4e5b-867b-58dc2b0ce100 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:19.726282 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44db28-aa2e-4ebb-8ca7-3a69dcc0aa3e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:19.750228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af70d7-b09c-4ff5-9e25-0eb74e225a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:19.750833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af70d7-b09c-4ff5-9e25-0eb74e225a3e None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:19.751145 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af70d7-b09c-4ff5-9e25-0eb74e225a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:19.751419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af70d7-b09c-4ff5-9e25-0eb74e225a3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:19.762484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6a65b-b7f5-4690-b688-ee70b3946199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:19.763470 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6a65b-b7f5-4690-b688-ee70b3946199 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:19.763749 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6a65b-b7f5-4690-b688-ee70b3946199 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:19.764321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6a65b-b7f5-4690-b688-ee70b3946199 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:19.769783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:19.775467 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a6a65b-b7f5-4690-b688-ee70b3946199 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:19.776949 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a6a65b-b7f5-4690-b688-ee70b3946199 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:19.791844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-82a6a65b-b7f5-4690-b688-ee70b3946199 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:19.791844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:19.791844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:19.791844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:19.791844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:19.791844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:19.791844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:19.791844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:19.791844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:19.791844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:19.791844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:19.791844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:19.791844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:19.791844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:19.791844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:19.791844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:19.792838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:19.799171 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:19.799171 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:19.799171 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:19.799171 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 337/649] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:19.799171 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6a65b-b7f5-4690-b688-ee70b3946199 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:19.828878 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af70d7-b09c-4ff5-9e25-0eb74e225a3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:19.830344 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af70d7-b09c-4ff5-9e25-0eb74e225a3e None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:19.863706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:19.912604 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6a65b-b7f5-4690-b688-ee70b3946199 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:19.934752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 313/650] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:54:19.934752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af70d7-b09c-4ff5-9e25-0eb74e225a3e None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:19.950860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e4bfe-6cd3-4abf-8fef-07007a374088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:19.951561 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e4bfe-6cd3-4abf-8fef-07007a374088 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:19.951836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e4bfe-6cd3-4abf-8fef-07007a374088 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:19.952100 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e4bfe-6cd3-4abf-8fef-07007a374088 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:19.952723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e4bfe-6cd3-4abf-8fef-07007a374088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:19.953019 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e4bfe-6cd3-4abf-8fef-07007a374088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:19.954370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e4bfe-6cd3-4abf-8fef-07007a374088 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:19.971248 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 338/651] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:19.971248 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e4bfe-6cd3-4abf-8fef-07007a374088 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:19.987727 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:20.040041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af70d7-b09c-4ff5-9e25-0eb74e225a3e None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:20.075179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c13d2-5434-461a-b65d-b4cca12cc79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:20.075837 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c13d2-5434-461a-b65d-b4cca12cc79d None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:20.076116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c13d2-5434-461a-b65d-b4cca12cc79d None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:20.076389 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c13d2-5434-461a-b65d-b4cca12cc79d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:20.077649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e4bfe-6cd3-4abf-8fef-07007a374088 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:20.084104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c13d2-5434-461a-b65d-b4cca12cc79d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:20.086239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c13d2-5434-461a-b65d-b4cca12cc79d None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:20.098777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-f07c13d2-5434-461a-b65d-b4cca12cc79d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:20.098777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:20.098777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:20.098777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:20.098777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:20.098777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:20.098777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:20.098777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:20.098777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:20.098777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:20.098777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:20.098777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:20.098777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:20.098777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:20.098777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:20.100548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:20.102037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:20.102037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:20.102037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:20.102037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:20.102037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:20.102037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:20.102037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:20.102037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:20.102037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 314/652] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:20.102037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c13d2-5434-461a-b65d-b4cca12cc79d None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:20.108961 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:20.121332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae946e5a-6c59-472c-9bf6-ae8b39a5ad3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:20.129857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae946e5a-6c59-472c-9bf6-ae8b39a5ad3c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:20.129857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae946e5a-6c59-472c-9bf6-ae8b39a5ad3c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:20.129857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae946e5a-6c59-472c-9bf6-ae8b39a5ad3c None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:20.129857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae946e5a-6c59-472c-9bf6-ae8b39a5ad3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:20.129857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae946e5a-6c59-472c-9bf6-ae8b39a5ad3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:20.129857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae946e5a-6c59-472c-9bf6-ae8b39a5ad3c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:20.138063 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 339/653] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:20.138063 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae946e5a-6c59-472c-9bf6-ae8b39a5ad3c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:20.218445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c13d2-5434-461a-b65d-b4cca12cc79d None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:20.227585 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:20.247338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae946e5a-6c59-472c-9bf6-ae8b39a5ad3c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:20.251917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a374589-eb98-4787-a50d-2e0055639e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:20.252546 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a374589-eb98-4787-a50d-2e0055639e58 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:20.252826 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a374589-eb98-4787-a50d-2e0055639e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:20.253099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a374589-eb98-4787-a50d-2e0055639e58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:20.263572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a374589-eb98-4787-a50d-2e0055639e58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:20.265041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a374589-eb98-4787-a50d-2e0055639e58 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:20.280774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-9a374589-eb98-4787-a50d-2e0055639e58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:20.280774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:20.280774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:20.280774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:20.280774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:20.280774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:20.280774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:20.280774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:20.280774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:20.280774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:20.280774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:20.280774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:20.280774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:20.280774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:20.280774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:20.280774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:20.281928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:20.282825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:20.282825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:20.282825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:20.283176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051ac46-1ca5-455a-9835-02a0197f2333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:20.283176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051ac46-1ca5-455a-9835-02a0197f2333 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:20.283321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 315/654] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:20.283403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051ac46-1ca5-455a-9835-02a0197f2333 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:20.283403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051ac46-1ca5-455a-9835-02a0197f2333 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:20.283538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a374589-eb98-4787-a50d-2e0055639e58 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:20.294259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051ac46-1ca5-455a-9835-02a0197f2333 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:20.295720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051ac46-1ca5-455a-9835-02a0197f2333 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:20.307263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-8051ac46-1ca5-455a-9835-02a0197f2333 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 06 07:54:20.307263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:20.307263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:20.307263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:20.307263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:20.307263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:20.307263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:20.307263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:20.307263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:20.307263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:20.307263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:20.307263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:20.307263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:20.307263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:20.307263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:20.307263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:20.308283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:20.309321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:20.309321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:20.309321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:20.309321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:20.309321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:20.309321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:20.309321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:20.309321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:20.309321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 06 07:54:20.309321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:20.309321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 340/655] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:54:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:20.309321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051ac46-1ca5-455a-9835-02a0197f2333 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:20.350814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:20.393596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a374589-eb98-4787-a50d-2e0055639e58 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:20.406842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051ac46-1ca5-455a-9835-02a0197f2333 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:20.427404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf4658-e29a-416f-94a1-5e393c240cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:20.428567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf4658-e29a-416f-94a1-5e393c240cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:20.428980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf4658-e29a-416f-94a1-5e393c240cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:20.429209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf4658-e29a-416f-94a1-5e393c240cf1 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:20.429793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf4658-e29a-416f-94a1-5e393c240cf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:20.430341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf4658-e29a-416f-94a1-5e393c240cf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:20.431579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf4658-e29a-416f-94a1-5e393c240cf1 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:20.440137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3aa55-1209-40ba-acfb-eafcf00f2e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:20.440773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3aa55-1209-40ba-acfb-eafcf00f2e69 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:20.441043 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3aa55-1209-40ba-acfb-eafcf00f2e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:20.441308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3aa55-1209-40ba-acfb-eafcf00f2e69 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:20.443417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 316/656] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:20.446118 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47de3d-e59d-409a-a96f-f3551654d3b2 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:20.446400 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47de3d-e59d-409a-a96f-f3551654d3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:20.446669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47de3d-e59d-409a-a96f-f3551654d3b2 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:20.451066 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 317/657] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:20.451066 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47de3d-e59d-409a-a96f-f3551654d3b2 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:20.451303 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f3aa55-1209-40ba-acfb-eafcf00f2e69 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:20.452641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f3aa55-1209-40ba-acfb-eafcf00f2e69 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:20.465535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-40f3aa55-1209-40ba-acfb-eafcf00f2e69 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 07:54:20.465535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:20.465535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:20.465535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:20.465535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:20.465535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:20.465535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:20.465535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:20.465535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:20.465535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:20.465535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:20.465535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:20.465535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:20.465535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:20.465535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:20.465535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:20.466663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:20.467721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:20.467721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:20.467721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:20.467721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:20.467721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:20.467721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:20.467721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:20.467721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:20.467721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 06 07:54:20.467721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:20.467721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 341/658] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:20.467721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3aa55-1209-40ba-acfb-eafcf00f2e69 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:20.476746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:20.562116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47de3d-e59d-409a-a96f-f3551654d3b2 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:20.568012 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3aa55-1209-40ba-acfb-eafcf00f2e69 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:20.570555 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:20.596564 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ea68a-1f88-4f45-b502-4e364b656734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:20.597160 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ea68a-1f88-4f45-b502-4e364b656734 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:20.597433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ea68a-1f88-4f45-b502-4e364b656734 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:20.597699 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ea68a-1f88-4f45-b502-4e364b656734 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:20.598329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ea68a-1f88-4f45-b502-4e364b656734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:20.598625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ea68a-1f88-4f45-b502-4e364b656734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:20.600023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ea68a-1f88-4f45-b502-4e364b656734 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:20.604061 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ea794-1eaf-422b-b771-2e6df001b099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:20.604652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ea794-1eaf-422b-b771-2e6df001b099 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:20.604925 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ea794-1eaf-422b-b771-2e6df001b099 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:20.605198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ea794-1eaf-422b-b771-2e6df001b099 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:20.605905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ea794-1eaf-422b-b771-2e6df001b099 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:20.606660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ea794-1eaf-422b-b771-2e6df001b099 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:20.608102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ea794-1eaf-422b-b771-2e6df001b099 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:20.610914 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 318/659] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:20.612014 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ea68a-1f88-4f45-b502-4e364b656734 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:20.626324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 342/660] 10.222.0.30 () {56 vars in 1137 bytes} [Thu Oct 6 07:54:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:20.627408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339893d7-6004-4ed2-8eed-e71d24d68171 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:20.627692 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339893d7-6004-4ed2-8eed-e71d24d68171 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:20.627960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339893d7-6004-4ed2-8eed-e71d24d68171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:20.646068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-339893d7-6004-4ed2-8eed-e71d24d68171 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:20.647871 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-339893d7-6004-4ed2-8eed-e71d24d68171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:20.691886 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:20.711725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ea68a-1f88-4f45-b502-4e364b656734 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:20.713650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 343/661] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:20.714444 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339893d7-6004-4ed2-8eed-e71d24d68171 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:20.754201 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432dc0ef-77b1-4d46-b9ca-a6eeaa13dba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:20.754873 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432dc0ef-77b1-4d46-b9ca-a6eeaa13dba2 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:20.755227 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432dc0ef-77b1-4d46-b9ca-a6eeaa13dba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:20.755393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432dc0ef-77b1-4d46-b9ca-a6eeaa13dba2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:20.756796 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432dc0ef-77b1-4d46-b9ca-a6eeaa13dba2 None admin] RBAC: Adding query filter params (role.id=9f0e36c2ed8843fbabd3a4cf30bc3725, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=209795285384486183cda455fe316bc9) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:20.757117 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432dc0ef-77b1-4d46-b9ca-a6eeaa13dba2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:20.758553 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432dc0ef-77b1-4d46-b9ca-a6eeaa13dba2 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:20.775318 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 319/662] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:20 2022] GET /identity/v3/role_assignments?user.id=209795285384486183cda455fe316bc9&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:20.775318 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432dc0ef-77b1-4d46-b9ca-a6eeaa13dba2 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:20.831475 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:20.844696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339893d7-6004-4ed2-8eed-e71d24d68171 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:20.886227 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0888663-1e2a-4d10-983e-05d8f736bb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:20.887015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0888663-1e2a-4d10-983e-05d8f736bb31 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:20.887359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0888663-1e2a-4d10-983e-05d8f736bb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:20.887694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0888663-1e2a-4d10-983e-05d8f736bb31 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:20.888498 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0888663-1e2a-4d10-983e-05d8f736bb31 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:20.888867 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0888663-1e2a-4d10-983e-05d8f736bb31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:20.890760 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0888663-1e2a-4d10-983e-05d8f736bb31 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:20.907703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432dc0ef-77b1-4d46-b9ca-a6eeaa13dba2 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:20.915119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 344/663] 10.222.0.30 () {56 vars in 1139 bytes} [Thu Oct 6 07:54:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:20.916187 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0888663-1e2a-4d10-983e-05d8f736bb31 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:20.961794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddabe4-3772-4203-8800-4bed1a6af3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:20.961794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddabe4-3772-4203-8800-4bed1a6af3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:20.962065 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddabe4-3772-4203-8800-4bed1a6af3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:20.962289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddabe4-3772-4203-8800-4bed1a6af3a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:20.972814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ddabe4-3772-4203-8800-4bed1a6af3a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:20.975653 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ddabe4-3772-4203-8800-4bed1a6af3a2 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:20.989573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-24ddabe4-3772-4203-8800-4bed1a6af3a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:20.989573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:20.989573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:20.989573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:20.989573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:20.989573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:20.989573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:20.989573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:20.989573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:20.989573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:20.989573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:20.989573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:20.989573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:20.989573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:20.989573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:20.990927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:20.992027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:20.992027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:20.992027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:20.992027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:20.992027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:20.992027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:20.992027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:20.992027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:20.992027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 320/664] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:20.992829 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddabe4-3772-4203-8800-4bed1a6af3a2 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:20.997850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:21.036589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0888663-1e2a-4d10-983e-05d8f736bb31 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:21.080639 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99adc3f-b927-470e-a5e7-e2ed2e9df15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:21.081243 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99adc3f-b927-470e-a5e7-e2ed2e9df15c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:21.081570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99adc3f-b927-470e-a5e7-e2ed2e9df15c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:21.081783 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99adc3f-b927-470e-a5e7-e2ed2e9df15c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:21.101912 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddabe4-3772-4203-8800-4bed1a6af3a2 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:21.102114 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:21.136476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f95ee0-7230-4315-9e27-a24db3a7f64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:21.137084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f95ee0-7230-4315-9e27-a24db3a7f64e None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:21.137421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f95ee0-7230-4315-9e27-a24db3a7f64e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:21.137693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f95ee0-7230-4315-9e27-a24db3a7f64e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:21.159164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99adc3f-b927-470e-a5e7-e2ed2e9df15c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:21.161116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99adc3f-b927-470e-a5e7-e2ed2e9df15c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:21.208960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f95ee0-7230-4315-9e27-a24db3a7f64e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:21.211160 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f95ee0-7230-4315-9e27-a24db3a7f64e None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:21.213815 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:21.264478 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 345/665] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:54:21.265354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99adc3f-b927-470e-a5e7-e2ed2e9df15c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:21.319984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 321/666] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:54:21.319984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f95ee0-7230-4315-9e27-a24db3a7f64e None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:21.322547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:21.378587 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99adc3f-b927-470e-a5e7-e2ed2e9df15c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:21.413634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e641d2-3083-4773-9fb8-7457a9c2e8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:21.414254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e641d2-3083-4773-9fb8-7457a9c2e8cf None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:21.414525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e641d2-3083-4773-9fb8-7457a9c2e8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:21.414791 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e641d2-3083-4773-9fb8-7457a9c2e8cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:21.420924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f95ee0-7230-4315-9e27-a24db3a7f64e None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:21.423646 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:21.429471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e641d2-3083-4773-9fb8-7457a9c2e8cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:21.435424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-99e641d2-3083-4773-9fb8-7457a9c2e8cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:21.435424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:21.435424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:21.435424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:21.435424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:21.435424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:21.435424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:21.435424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:21.435424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:21.435424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:21.435424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:21.435424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:21.435424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:21.435424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:21.435424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:21.435424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:21.444772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:21.444772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:21.444772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:21.444772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:21.444772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:21.444772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:21.444772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:21.444772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:21.444772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:21.444772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:21.444772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:21.444772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:21.444772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:21.444772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:21.444772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:21.444772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:21.445630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:21.445630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:21.445630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:21.445630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:21.445630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:21.445630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:21.445630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:21.445630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:21.445630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:21.445630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:21.445630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:21.445630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:21.445630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 346/667] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:21.445630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e641d2-3083-4773-9fb8-7457a9c2e8cf None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:21.464298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa805e9-625e-4a67-8eb8-cd2c616a3d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:21.464973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa805e9-625e-4a67-8eb8-cd2c616a3d40 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:21.465257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa805e9-625e-4a67-8eb8-cd2c616a3d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:21.465531 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa805e9-625e-4a67-8eb8-cd2c616a3d40 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:21.466229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa805e9-625e-4a67-8eb8-cd2c616a3d40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:21.466536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa805e9-625e-4a67-8eb8-cd2c616a3d40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:21.467995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa805e9-625e-4a67-8eb8-cd2c616a3d40 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:21.479831 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 322/668] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:21.480625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa805e9-625e-4a67-8eb8-cd2c616a3d40 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:21.556536 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e641d2-3083-4773-9fb8-7457a9c2e8cf None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:21.566089 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:21.587655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa805e9-625e-4a67-8eb8-cd2c616a3d40 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:21.592768 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23ae78-e7b5-4746-a61d-4e62901fb2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:21.593397 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23ae78-e7b5-4746-a61d-4e62901fb2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:21.593670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23ae78-e7b5-4746-a61d-4e62901fb2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:21.593936 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23ae78-e7b5-4746-a61d-4e62901fb2c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:21.607659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23ae78-e7b5-4746-a61d-4e62901fb2c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:21.609422 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23ae78-e7b5-4746-a61d-4e62901fb2c3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:21.622230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906eba8-bfe9-4b46-83d5-cf4b99ae2753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:21.622865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906eba8-bfe9-4b46-83d5-cf4b99ae2753 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:21.623173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906eba8-bfe9-4b46-83d5-cf4b99ae2753 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:21.623712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906eba8-bfe9-4b46-83d5-cf4b99ae2753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:21.626168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-dd23ae78-e7b5-4746-a61d-4e62901fb2c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:21.626168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:21.626168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:21.626168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:21.626168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:21.626168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:21.626168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:21.626168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:21.626168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:21.626168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:21.626168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:21.626168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:21.626168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:21.626168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:21.626168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:21.626168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:21.627110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:21.628008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:21.628008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:21.628008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:21.628008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 347/669] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:21.628230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23ae78-e7b5-4746-a61d-4e62901fb2c3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:21.640458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906eba8-bfe9-4b46-83d5-cf4b99ae2753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:21.641851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906eba8-bfe9-4b46-83d5-cf4b99ae2753 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:21.655795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-b906eba8-bfe9-4b46-83d5-cf4b99ae2753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:21.655795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:21.655795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:21.655795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:21.655795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:21.655795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:21.655795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:21.655795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:21.655795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:21.655795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:21.655795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:21.655795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:21.655795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:21.655795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:21.655795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:21.655795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:21.656758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:21.657651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:21.657651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:21.657651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:21.657651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 323/670] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:21.657651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906eba8-bfe9-4b46-83d5-cf4b99ae2753 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:21.675920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:21.740994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23ae78-e7b5-4746-a61d-4e62901fb2c3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:21.760805 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906eba8-bfe9-4b46-83d5-cf4b99ae2753 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:21.781346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7d609-737b-482c-a97e-9ce9c71572e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:21.781949 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7d609-737b-482c-a97e-9ce9c71572e6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:21.782226 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7d609-737b-482c-a97e-9ce9c71572e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:21.782953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7d609-737b-482c-a97e-9ce9c71572e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:21.783638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7d609-737b-482c-a97e-9ce9c71572e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:21.784096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7d609-737b-482c-a97e-9ce9c71572e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:21.785467 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7d609-737b-482c-a97e-9ce9c71572e6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:21.794412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 348/671] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:21.795155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7d609-737b-482c-a97e-9ce9c71572e6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:21.800731 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f99cb7-4ed0-4215-98f6-04cf71746570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:21.801344 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f99cb7-4ed0-4215-98f6-04cf71746570 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:21.801624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f99cb7-4ed0-4215-98f6-04cf71746570 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:21.801896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f99cb7-4ed0-4215-98f6-04cf71746570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:21.802623 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f99cb7-4ed0-4215-98f6-04cf71746570 None admin] RBAC: Adding query filter params (role.id=9f0e36c2ed8843fbabd3a4cf30bc3725, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=a899c6dabca948269f6c1370425f244b) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:21.802972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f99cb7-4ed0-4215-98f6-04cf71746570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:21.804442 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f99cb7-4ed0-4215-98f6-04cf71746570 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:21.810433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 324/672] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:21 2022] GET /identity/v3/role_assignments?user.id=a899c6dabca948269f6c1370425f244b&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:21.811248 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f99cb7-4ed0-4215-98f6-04cf71746570 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:21.838743 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:21.929833 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7d609-737b-482c-a97e-9ce9c71572e6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:21.932757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:21.954520 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f99cb7-4ed0-4215-98f6-04cf71746570 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:21.982750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580646e-bb50-4ea7-9710-1af03ef820f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:21.983410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580646e-bb50-4ea7-9710-1af03ef820f7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:21.983691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580646e-bb50-4ea7-9710-1af03ef820f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:21.984429 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580646e-bb50-4ea7-9710-1af03ef820f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:21.985096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580646e-bb50-4ea7-9710-1af03ef820f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:21.985667 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580646e-bb50-4ea7-9710-1af03ef820f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:21.987425 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580646e-bb50-4ea7-9710-1af03ef820f7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:21.998694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe11c28-a6b2-4a07-9d45-2c382d3ea415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:21.999577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe11c28-a6b2-4a07-9d45-2c382d3ea415 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:21.999856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe11c28-a6b2-4a07-9d45-2c382d3ea415 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:22.000130 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe11c28-a6b2-4a07-9d45-2c382d3ea415 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:22.001574 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe11c28-a6b2-4a07-9d45-2c382d3ea415 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:22.001877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe11c28-a6b2-4a07-9d45-2c382d3ea415 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:22.001978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 349/673] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:22.002780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580646e-bb50-4ea7-9710-1af03ef820f7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:22.003280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe11c28-a6b2-4a07-9d45-2c382d3ea415 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:22.020151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 325/674] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:22.021013 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe11c28-a6b2-4a07-9d45-2c382d3ea415 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:22.108036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:22.132737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580646e-bb50-4ea7-9710-1af03ef820f7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:22.157407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe11c28-a6b2-4a07-9d45-2c382d3ea415 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:22.169953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf48cef-24fa-4114-8a6e-ee15d998c345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:22.170702 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf48cef-24fa-4114-8a6e-ee15d998c345 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:22.171002 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf48cef-24fa-4114-8a6e-ee15d998c345 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:22.171280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf48cef-24fa-4114-8a6e-ee15d998c345 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:22.182822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf48cef-24fa-4114-8a6e-ee15d998c345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:22.184257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf48cef-24fa-4114-8a6e-ee15d998c345 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:22.199814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f5ad5-3e38-4201-81c7-ed0a23599e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:22.200450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f5ad5-3e38-4201-81c7-ed0a23599e1a None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:22.200688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5cf48cef-24fa-4114-8a6e-ee15d998c345 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:22.200688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:22.200688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:22.200688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:22.200688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:22.200688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:22.200688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:22.200688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:22.200688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:22.200688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:22.200688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:22.200688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:22.200688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:22.200688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:22.200688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:22.200688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:22.201601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:22.202873 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:22.202873 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:22.202873 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:22.203115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f5ad5-3e38-4201-81c7-ed0a23599e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:22.203115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f5ad5-3e38-4201-81c7-ed0a23599e1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:22.203245 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 350/675] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:22.203245 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf48cef-24fa-4114-8a6e-ee15d998c345 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:22.214218 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f5ad5-3e38-4201-81c7-ed0a23599e1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:22.215742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f5ad5-3e38-4201-81c7-ed0a23599e1a None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:22.219428 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:22.232710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-219f5ad5-3e38-4201-81c7-ed0a23599e1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:22.232710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:22.232710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:22.232710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:22.232710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:22.232710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:22.232710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:22.232710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:22.232710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:22.232710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:22.232710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:22.232710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:22.232710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:22.232710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:22.232710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:22.233856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:22.234972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:22.234972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:22.234972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:22.234972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:22.234972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:22.234972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:22.234972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:22.234972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:22.234972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 326/676] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:22.239182 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f5ad5-3e38-4201-81c7-ed0a23599e1a None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:22.349380 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf48cef-24fa-4114-8a6e-ee15d998c345 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:22.366775 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f5ad5-3e38-4201-81c7-ed0a23599e1a None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:22.398318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2c283-83e0-4f78-bee2-8c2e73bc64ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:22.399518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2c283-83e0-4f78-bee2-8c2e73bc64ed None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:22.400039 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2c283-83e0-4f78-bee2-8c2e73bc64ed None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:22.400460 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2c283-83e0-4f78-bee2-8c2e73bc64ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:22.402693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:22.403282 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75448028-d483-49c7-a4d0-0b2647a9fd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:22.403951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75448028-d483-49c7-a4d0-0b2647a9fd61 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:22.404527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75448028-d483-49c7-a4d0-0b2647a9fd61 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:22.404662 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75448028-d483-49c7-a4d0-0b2647a9fd61 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:22.405486 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75448028-d483-49c7-a4d0-0b2647a9fd61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:22.405889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75448028-d483-49c7-a4d0-0b2647a9fd61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:22.407379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75448028-d483-49c7-a4d0-0b2647a9fd61 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:22.410399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2c283-83e0-4f78-bee2-8c2e73bc64ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:22.412055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2c283-83e0-4f78-bee2-8c2e73bc64ed None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:22.420131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 327/677] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:22.421132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75448028-d483-49c7-a4d0-0b2647a9fd61 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:22.432138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e8a2c283-83e0-4f78-bee2-8c2e73bc64ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:22.432138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:22.432138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:22.432138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:22.432138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:22.432138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:22.432138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:22.432138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:22.432138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:22.432138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:22.432138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:22.432138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:22.432138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:22.432138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:22.432138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:22.433214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:22.434130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:22.434130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:22.434130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:22.434130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:22.434130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:22.434130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:22.434130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:22.434130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:22.434130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 351/678] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:22.437153 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2c283-83e0-4f78-bee2-8c2e73bc64ed None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:22.489567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:22.534267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75448028-d483-49c7-a4d0-0b2647a9fd61 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:22.573347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588620d3-14e5-4836-9150-f42bd44d619d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:22.574176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588620d3-14e5-4836-9150-f42bd44d619d None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:22.574679 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588620d3-14e5-4836-9150-f42bd44d619d None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:22.574969 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588620d3-14e5-4836-9150-f42bd44d619d None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:22.575667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588620d3-14e5-4836-9150-f42bd44d619d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:22.575966 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588620d3-14e5-4836-9150-f42bd44d619d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:22.577914 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588620d3-14e5-4836-9150-f42bd44d619d None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:22.583752 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2c283-83e0-4f78-bee2-8c2e73bc64ed None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:22.592559 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 328/679] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:22.593954 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588620d3-14e5-4836-9150-f42bd44d619d None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:22.631995 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0aabb-b72d-4868-863e-bea6d19ed1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:22.633137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0aabb-b72d-4868-863e-bea6d19ed1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:22.633585 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0aabb-b72d-4868-863e-bea6d19ed1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:22.634459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0aabb-b72d-4868-863e-bea6d19ed1c1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:22.644924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0aabb-b72d-4868-863e-bea6d19ed1c1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:22.646445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0aabb-b72d-4868-863e-bea6d19ed1c1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:22.658367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ace0aabb-b72d-4868-863e-bea6d19ed1c1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 07:54:22.658367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:22.658367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:22.658367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:22.658367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:22.658367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:22.658367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:22.658367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:22.658367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:22.658367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:22.658367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:22.658367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:22.658367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:22.658367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:22.658367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:22.658367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:22.659580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:22.660847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:22.660847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:22.660847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:22.660847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:22.660847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:22.660847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:22.660847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:22.660847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:22.660847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 07:54:22.660847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:22.666728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 352/680] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:54:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:22.667648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0aabb-b72d-4868-863e-bea6d19ed1c1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:22.716542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588620d3-14e5-4836-9150-f42bd44d619d None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:22.724055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:22.739876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:22.755562 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55c6af-203f-4013-b616-a6dc54deddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:22.756448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55c6af-203f-4013-b616-a6dc54deddec None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:22.757040 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55c6af-203f-4013-b616-a6dc54deddec None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:22.757530 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55c6af-203f-4013-b616-a6dc54deddec None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:22.759398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55c6af-203f-4013-b616-a6dc54deddec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:22.759879 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55c6af-203f-4013-b616-a6dc54deddec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:22.761589 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55c6af-203f-4013-b616-a6dc54deddec None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:22.771225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0aabb-b72d-4868-863e-bea6d19ed1c1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:22.777149 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 329/681] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:22.778367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55c6af-203f-4013-b616-a6dc54deddec None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:22.807854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608de70c-bd1a-4cbe-9d30-900f2ee307ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:22.808298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608de70c-bd1a-4cbe-9d30-900f2ee307ac None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:22.808575 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608de70c-bd1a-4cbe-9d30-900f2ee307ac None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:22.808873 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608de70c-bd1a-4cbe-9d30-900f2ee307ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:22.809721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608de70c-bd1a-4cbe-9d30-900f2ee307ac None admin] RBAC: Adding query filter params (role.id=9f0e36c2ed8843fbabd3a4cf30bc3725, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=30909d152a5c4a1fac6c1a2765868177) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:22.810028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608de70c-bd1a-4cbe-9d30-900f2ee307ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:22.811513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608de70c-bd1a-4cbe-9d30-900f2ee307ac None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:22.819128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 353/682] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:22 2022] GET /identity/v3/role_assignments?user.id=30909d152a5c4a1fac6c1a2765868177&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:22.820323 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608de70c-bd1a-4cbe-9d30-900f2ee307ac None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:22.905813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55c6af-203f-4013-b616-a6dc54deddec None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:22.946822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2c36e-8def-4162-be44-541226f40a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:22.947725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2c36e-8def-4162-be44-541226f40a44 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:22.948177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2c36e-8def-4162-be44-541226f40a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:22.948696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2c36e-8def-4162-be44-541226f40a44 None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:22.949696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2c36e-8def-4162-be44-541226f40a44 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:22.950154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2c36e-8def-4162-be44-541226f40a44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:22.951844 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2c36e-8def-4162-be44-541226f40a44 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:22.973358 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 330/683] 10.222.0.30 () {56 vars in 1135 bytes} [Thu Oct 6 07:54:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:22.980420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2c36e-8def-4162-be44-541226f40a44 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:22.985560 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608de70c-bd1a-4cbe-9d30-900f2ee307ac None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:23.032763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:23.035415 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:23.060319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22ed49-502b-4f29-84a8-63fc958de7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:23.060876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22ed49-502b-4f29-84a8-63fc958de7bf None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:23.061151 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22ed49-502b-4f29-84a8-63fc958de7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:23.061526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22ed49-502b-4f29-84a8-63fc958de7bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:23.062299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e22ed49-502b-4f29-84a8-63fc958de7bf None admin] RBAC: Adding query filter params (role.id=9f0e36c2ed8843fbabd3a4cf30bc3725, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=bf6798f89acf44e897c481924edd9047) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:23.062610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e22ed49-502b-4f29-84a8-63fc958de7bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:23.065581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e22ed49-502b-4f29-84a8-63fc958de7bf None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:23.110877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 354/684] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:22 2022] GET /identity/v3/role_assignments?user.id=bf6798f89acf44e897c481924edd9047&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:23.117922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2c36e-8def-4162-be44-541226f40a44 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:23.166308 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212dc2d9-4597-45b2-95d3-e3c9a140a911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:23.166965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dc2d9-4597-45b2-95d3-e3c9a140a911 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:23.167313 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dc2d9-4597-45b2-95d3-e3c9a140a911 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:23.167594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dc2d9-4597-45b2-95d3-e3c9a140a911 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:23.180627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06adaa-23a6-4137-bcd3-d51c9604cdcf None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:23.180936 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06adaa-23a6-4137-bcd3-d51c9604cdcf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:23.181176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06adaa-23a6-4137-bcd3-d51c9604cdcf None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:23.182866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 355/685] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:23.192940 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f24331-b066-4d24-a42d-e701a492eb41 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:23.193266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f24331-b066-4d24-a42d-e701a492eb41 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:23.193527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f24331-b066-4d24-a42d-e701a492eb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:23.216461 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-47f24331-b066-4d24-a42d-e701a492eb41 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:23.217757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-47f24331-b066-4d24-a42d-e701a492eb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:23.254097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212dc2d9-4597-45b2-95d3-e3c9a140a911 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:23.255820 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212dc2d9-4597-45b2-95d3-e3c9a140a911 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:23.301757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:23.311092 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 356/686] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:54:23.372240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 331/687] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:54:23.378201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f24331-b066-4d24-a42d-e701a492eb41 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:23.608627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f24331-b066-4d24-a42d-e701a492eb41 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:23.626598 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:23.630469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212dc2d9-4597-45b2-95d3-e3c9a140a911 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:23.670267 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4ab1c-064d-4aab-9563-99a036f0d1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:23.670908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4ab1c-064d-4aab-9563-99a036f0d1ef None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:23.671210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4ab1c-064d-4aab-9563-99a036f0d1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:23.671474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4ab1c-064d-4aab-9563-99a036f0d1ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:23.682597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4ab1c-064d-4aab-9563-99a036f0d1ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:23.684155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4ab1c-064d-4aab-9563-99a036f0d1ef None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:23.699797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ceb4ab1c-064d-4aab-9563-99a036f0d1ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:23.699797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:23.699797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:23.699797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:23.699797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:23.699797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:23.699797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:23.699797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:23.699797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:23.699797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:23.699797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:23.699797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:23.699797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:23.699797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:23.699797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:23.699797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:23.700781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:23.701888 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:23.701888 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:23.701888 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:23.701888 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 357/688] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:23.707877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4ab1c-064d-4aab-9563-99a036f0d1ef None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:23.820398 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4ab1c-064d-4aab-9563-99a036f0d1ef None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:23.857247 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2750db7-0b16-4566-8531-9be3d3b4130a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:23.857852 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2750db7-0b16-4566-8531-9be3d3b4130a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:23.858124 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2750db7-0b16-4566-8531-9be3d3b4130a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:23.858390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2750db7-0b16-4566-8531-9be3d3b4130a None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:23.859238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2750db7-0b16-4566-8531-9be3d3b4130a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:23.859480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2750db7-0b16-4566-8531-9be3d3b4130a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:23.860842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2750db7-0b16-4566-8531-9be3d3b4130a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:23.874501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 358/689] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:23.880557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2750db7-0b16-4566-8531-9be3d3b4130a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:23.895893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212dc2d9-4597-45b2-95d3-e3c9a140a911 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:23.935310 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:23.953733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6728a1b-d060-4c6a-93a5-7532e3a3455f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:23.954388 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6728a1b-d060-4c6a-93a5-7532e3a3455f None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:23.954665 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6728a1b-d060-4c6a-93a5-7532e3a3455f None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:23.954931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6728a1b-d060-4c6a-93a5-7532e3a3455f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:23.976603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:23.983896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6728a1b-d060-4c6a-93a5-7532e3a3455f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:23.988694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-f6728a1b-d060-4c6a-93a5-7532e3a3455f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:54:23.988694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:23.988694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:23.988694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:23.988694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:23.988694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:23.988694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:23.988694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:23.988694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:23.988694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:23.988694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:23.988694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:23.988694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:23.988694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:23.988694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:23.988694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:23.991429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:23.991429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:23.991429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:23.991429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:23.991429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:23.991429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:23.991429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:23.991429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:23.991429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:23.991429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:23.991429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:23.991429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:23.991429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:23.991429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:23.991429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:23.991429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:23.992289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:23.992289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:23.992289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:23.992289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:23.992289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:23.992289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:23.992289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:23.992289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:23.992289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:23.992289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:23.992289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:54:23.992289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:23.992289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 332/690] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:54:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:23.996538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6728a1b-d060-4c6a-93a5-7532e3a3455f None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:24.034615 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2750db7-0b16-4566-8531-9be3d3b4130a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:24.084622 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10408ab2-b79f-42f3-a1f9-128c471c597e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:24.085258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10408ab2-b79f-42f3-a1f9-128c471c597e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:24.085532 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10408ab2-b79f-42f3-a1f9-128c471c597e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:24.085798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10408ab2-b79f-42f3-a1f9-128c471c597e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:24.101731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10408ab2-b79f-42f3-a1f9-128c471c597e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:24.103211 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10408ab2-b79f-42f3-a1f9-128c471c597e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:24.130134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-10408ab2-b79f-42f3-a1f9-128c471c597e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:24.130134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:24.130134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:24.130134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:24.130134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:24.130134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:24.130134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:24.130134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:24.130134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:24.130134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:24.130134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:24.130134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:24.130134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:24.130134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:24.130134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:24.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:24.132659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:24.132659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:24.132659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:24.132659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:24.132659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:24.132659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:24.132659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:24.132659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:24.134772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 359/691] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:24.138087 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10408ab2-b79f-42f3-a1f9-128c471c597e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:24.173560 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6728a1b-d060-4c6a-93a5-7532e3a3455f None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:24.210467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a5143-41c1-43a8-b5ce-039244f2afeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:24.211098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a5143-41c1-43a8-b5ce-039244f2afeb None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:24.211370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a5143-41c1-43a8-b5ce-039244f2afeb None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:24.211642 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a5143-41c1-43a8-b5ce-039244f2afeb None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:24.212301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a5143-41c1-43a8-b5ce-039244f2afeb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:24.212596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a5143-41c1-43a8-b5ce-039244f2afeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:24.213974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a5143-41c1-43a8-b5ce-039244f2afeb None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:24.222845 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 333/692] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:54:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:24.230504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a5143-41c1-43a8-b5ce-039244f2afeb None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:24.307302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:24.326759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10408ab2-b79f-42f3-a1f9-128c471c597e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:24.361622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:24.368693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be59b5b-7564-4126-b261-fb6974979125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:24.369542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be59b5b-7564-4126-b261-fb6974979125 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:24.382994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be59b5b-7564-4126-b261-fb6974979125 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:24.382994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be59b5b-7564-4126-b261-fb6974979125 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:24.382994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be59b5b-7564-4126-b261-fb6974979125 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:24.382994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be59b5b-7564-4126-b261-fb6974979125 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:24.382994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be59b5b-7564-4126-b261-fb6974979125 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:24.396373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 360/693] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:24.402258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be59b5b-7564-4126-b261-fb6974979125 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:24.420604 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a5143-41c1-43a8-b5ce-039244f2afeb None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:24.457846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef097b4-e060-40d1-90a6-1e7ea00e5150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:24.459079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef097b4-e060-40d1-90a6-1e7ea00e5150 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:24.459684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef097b4-e060-40d1-90a6-1e7ea00e5150 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:24.460482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef097b4-e060-40d1-90a6-1e7ea00e5150 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:24.472541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef097b4-e060-40d1-90a6-1e7ea00e5150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:24.474320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef097b4-e060-40d1-90a6-1e7ea00e5150 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:24.491609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-5ef097b4-e060-40d1-90a6-1e7ea00e5150 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:24.491609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:24.491609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:24.491609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:24.491609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:24.491609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:24.491609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:24.491609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:24.491609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:24.491609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:24.491609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:24.491609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:24.491609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:24.491609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:24.491609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:24.491609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:24.492594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:24.493488 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:24.493488 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:24.493488 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:24.495704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 334/694] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:24.497155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51c862-6a1c-4b16-b54f-1b8cc6308abc None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:24.497696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51c862-6a1c-4b16-b54f-1b8cc6308abc None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:24.498512 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51c862-6a1c-4b16-b54f-1b8cc6308abc None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:24.500533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 335/695] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:24.501503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51c862-6a1c-4b16-b54f-1b8cc6308abc None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:24.523347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be59b5b-7564-4126-b261-fb6974979125 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:24.567071 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a36da-2000-4c06-8731-8dec6c4dd770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:24.567719 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a36da-2000-4c06-8731-8dec6c4dd770 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:24.568043 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a36da-2000-4c06-8731-8dec6c4dd770 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:24.568313 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a36da-2000-4c06-8731-8dec6c4dd770 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:24.569201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a36da-2000-4c06-8731-8dec6c4dd770 None admin] RBAC: Adding query filter params (role.id=9f0e36c2ed8843fbabd3a4cf30bc3725, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=3fa1c430c9f7416eaf1d17300b827e05) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:24.569505 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a36da-2000-4c06-8731-8dec6c4dd770 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:24.571031 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a36da-2000-4c06-8731-8dec6c4dd770 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:24.577273 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 361/696] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:24 2022] GET /identity/v3/role_assignments?user.id=3fa1c430c9f7416eaf1d17300b827e05&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:24.578499 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1f521-ab2e-4e83-96ac-ccd85214fc96 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:24.578782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1f521-ab2e-4e83-96ac-ccd85214fc96 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:24.579082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1f521-ab2e-4e83-96ac-ccd85214fc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:24.603636 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-33e1f521-ab2e-4e83-96ac-ccd85214fc96 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:24.603636 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-33e1f521-ab2e-4e83-96ac-ccd85214fc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:24.604606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:24.656864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51c862-6a1c-4b16-b54f-1b8cc6308abc None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:24.674369 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:24.701234 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd454fb-28a2-4e95-b522-699252aeb0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:24.702351 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd454fb-28a2-4e95-b522-699252aeb0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:24.702890 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd454fb-28a2-4e95-b522-699252aeb0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:24.703428 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd454fb-28a2-4e95-b522-699252aeb0e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:24.704269 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd454fb-28a2-4e95-b522-699252aeb0e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:24.704924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd454fb-28a2-4e95-b522-699252aeb0e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:24.707054 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd454fb-28a2-4e95-b522-699252aeb0e3 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:24.723378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 336/697] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:24.729986 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd454fb-28a2-4e95-b522-699252aeb0e3 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:24.771396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 362/698] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:24.870250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd454fb-28a2-4e95-b522-699252aeb0e3 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:24.907287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dddd5-de38-4c62-ba3e-0c114d065350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:24.907880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dddd5-de38-4c62-ba3e-0c114d065350 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:24.908119 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dddd5-de38-4c62-ba3e-0c114d065350 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:24.908382 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2dddd5-de38-4c62-ba3e-0c114d065350 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:24.916160 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dddd5-de38-4c62-ba3e-0c114d065350 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:24.917591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2dddd5-de38-4c62-ba3e-0c114d065350 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:24.929817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-1c2dddd5-de38-4c62-ba3e-0c114d065350 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:54:24.929817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:24.929817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:24.929817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:24.929817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:24.929817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:24.929817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:24.929817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:24.929817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:24.929817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:24.929817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:24.929817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:24.929817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:24.929817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:24.929817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:24.929817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:24.930832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:24.931840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:24.931840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:24.931840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:24.931840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:24.931840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:24.931840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:24.931840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:24.931840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:24.931840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:54:24.931840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:24.931840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 337/699] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Oct 6 07:54:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:24.936581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1f521-ab2e-4e83-96ac-ccd85214fc96 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:25.058618 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1f521-ab2e-4e83-96ac-ccd85214fc96 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:25.102377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1517bf-0554-4541-81f8-b54888327890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:25.103005 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1517bf-0554-4541-81f8-b54888327890 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:25.103281 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1517bf-0554-4541-81f8-b54888327890 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:25.103546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1517bf-0554-4541-81f8-b54888327890 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:25.104275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1517bf-0554-4541-81f8-b54888327890 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:25.104577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1517bf-0554-4541-81f8-b54888327890 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:25.105999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1517bf-0554-4541-81f8-b54888327890 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:25.117316 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:25.125042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dddd5-de38-4c62-ba3e-0c114d065350 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:25.132348 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 363/700] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Oct 6 07:54:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:25.139980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1517bf-0554-4541-81f8-b54888327890 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:25.164049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:25.251844 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2dddd5-de38-4c62-ba3e-0c114d065350 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:25.314118 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1517bf-0554-4541-81f8-b54888327890 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:25.317482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5276e-146f-4809-a336-345b9114c181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:25.318123 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5276e-146f-4809-a336-345b9114c181 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:25.318460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5276e-146f-4809-a336-345b9114c181 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:25.319691 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5276e-146f-4809-a336-345b9114c181 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:25.319817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5276e-146f-4809-a336-345b9114c181 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:25.321179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5276e-146f-4809-a336-345b9114c181 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-46e5276e-146f-4809-a336-345b9114c181 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:25.341667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:25.342752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:25.342752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:25.342752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:25.342752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:25.342752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:25.342752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:25.342752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:25.342752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:25.342752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:25.342752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:25.342752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:25.342752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:25.342752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:25.342752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:25.342752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:25.342752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:25.344354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:25.344354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:25.344354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:25.344354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:25.344354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:25.344354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:25.344354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:25.344354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:25.344354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:25.344354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:25.344354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:25.344354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:25.344354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 07:54:25.344354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:25.345094 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 338/701] 10.222.0.30 () {56 vars in 1129 bytes} [Thu Oct 6 07:54:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:25.351035 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5276e-146f-4809-a336-345b9114c181 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:25.383577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccb2fa-5224-4988-8ffd-a4aab12ccdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:25.384234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccb2fa-5224-4988-8ffd-a4aab12ccdff None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:25.384511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccb2fa-5224-4988-8ffd-a4aab12ccdff None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:25.384773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccb2fa-5224-4988-8ffd-a4aab12ccdff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:25.482094 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:25.490302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ccb2fa-5224-4988-8ffd-a4aab12ccdff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:25.491931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ccb2fa-5224-4988-8ffd-a4aab12ccdff None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:25.550915 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:25.560241 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5276e-146f-4809-a336-345b9114c181 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:25.601008 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba90f2-41c9-4bad-abde-d11a9db4b283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:25.601867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba90f2-41c9-4bad-abde-d11a9db4b283 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:25.602245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba90f2-41c9-4bad-abde-d11a9db4b283 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:25.602622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba90f2-41c9-4bad-abde-d11a9db4b283 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:25.603698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba90f2-41c9-4bad-abde-d11a9db4b283 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:25.604112 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba90f2-41c9-4bad-abde-d11a9db4b283 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:25.605935 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba90f2-41c9-4bad-abde-d11a9db4b283 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:25.611719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 339/702] 10.222.0.30 () {56 vars in 1124 bytes} [Thu Oct 6 07:54:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:25.617974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba90f2-41c9-4bad-abde-d11a9db4b283 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:25.697278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 364/703] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:54:25.704184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccb2fa-5224-4988-8ffd-a4aab12ccdff None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:25.828151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba90f2-41c9-4bad-abde-d11a9db4b283 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:25.874603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10ffeb-2e14-4d25-87e4-69b861204869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:25.875489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10ffeb-2e14-4d25-87e4-69b861204869 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:25.875913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10ffeb-2e14-4d25-87e4-69b861204869 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:25.876329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10ffeb-2e14-4d25-87e4-69b861204869 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:25.877102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10ffeb-2e14-4d25-87e4-69b861204869 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:25.877571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10ffeb-2e14-4d25-87e4-69b861204869 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:25.879122 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10ffeb-2e14-4d25-87e4-69b861204869 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:25.880181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:25.886748 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 340/704] 10.222.0.30 () {56 vars in 1124 bytes} [Thu Oct 6 07:54:25 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:25.888380 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e2456-202d-4ad6-a329-1b3a730d140a None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:25.901409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccb2fa-5224-4988-8ffd-a4aab12ccdff None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:25.901569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e2456-202d-4ad6-a329-1b3a730d140a None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:25.901569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e2456-202d-4ad6-a329-1b3a730d140a None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:25.901569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 341/705] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:25.904003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7616f89-7fda-471e-81dd-514c30643c6c None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:25.905014 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7616f89-7fda-471e-81dd-514c30643c6c None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:25.905461 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7616f89-7fda-471e-81dd-514c30643c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:25.934920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-d7616f89-7fda-471e-81dd-514c30643c6c None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:25.936250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d7616f89-7fda-471e-81dd-514c30643c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:25.945908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df36815-8d49-4689-ac3d-19157306d82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:25.946567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df36815-8d49-4689-ac3d-19157306d82f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:25.946839 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df36815-8d49-4689-ac3d-19157306d82f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:25.947137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df36815-8d49-4689-ac3d-19157306d82f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:25.962219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df36815-8d49-4689-ac3d-19157306d82f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:25.964527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:25.965884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df36815-8d49-4689-ac3d-19157306d82f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:25.984366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-4df36815-8d49-4689-ac3d-19157306d82f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:25.984366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:25.984366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:25.984366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:25.984366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:25.984366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:25.984366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:25.984366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:25.984366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:25.984366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:25.984366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:25.984366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:25.984366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:25.984366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:25.984366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:25.984366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:25.985412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:25.986305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:25.986305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:25.986305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:25.989297 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 365/706] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:25.991675 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1efd41-372b-4721-b5c4-e412d62cd1c8 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:25.992143 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1efd41-372b-4721-b5c4-e412d62cd1c8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:25.992553 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1efd41-372b-4721-b5c4-e412d62cd1c8 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:25.999056 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 366/707] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:25.999722 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1efd41-372b-4721-b5c4-e412d62cd1c8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:26.038588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 342/708] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:26.044402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08084e1-5605-4e1f-836f-c7c84fa2e984 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:26.044402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08084e1-5605-4e1f-836f-c7c84fa2e984 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:26.044402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08084e1-5605-4e1f-836f-c7c84fa2e984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:26.064409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-d08084e1-5605-4e1f-836f-c7c84fa2e984 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:26.066576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d08084e1-5605-4e1f-836f-c7c84fa2e984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:26.145918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 343/709] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:26.156322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1efd41-372b-4721-b5c4-e412d62cd1c8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:26.201007 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fee2b0-9852-4401-a3ec-f5db0c9b5219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:26.201613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fee2b0-9852-4401-a3ec-f5db0c9b5219 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:26.201889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fee2b0-9852-4401-a3ec-f5db0c9b5219 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:26.202159 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fee2b0-9852-4401-a3ec-f5db0c9b5219 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:26.202796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fee2b0-9852-4401-a3ec-f5db0c9b5219 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:26.203132 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fee2b0-9852-4401-a3ec-f5db0c9b5219 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:26.204610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fee2b0-9852-4401-a3ec-f5db0c9b5219 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:26.218053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 367/710] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:26.223346 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08084e1-5605-4e1f-836f-c7c84fa2e984 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:26.316811 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:26.333498 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08084e1-5605-4e1f-836f-c7c84fa2e984 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:26.374729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fee2b0-9852-4401-a3ec-f5db0c9b5219 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:26.376566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51518219-77a8-4f70-968a-d569ad77d8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:26.377202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51518219-77a8-4f70-968a-d569ad77d8ab None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:26.377785 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51518219-77a8-4f70-968a-d569ad77d8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:26.378064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51518219-77a8-4f70-968a-d569ad77d8ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:26.387067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51518219-77a8-4f70-968a-d569ad77d8ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:26.389019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51518219-77a8-4f70-968a-d569ad77d8ab None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:26.399363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:26.402599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-51518219-77a8-4f70-968a-d569ad77d8ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:26.402599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:26.402599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:26.402599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:26.402599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:26.402599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:26.402599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:26.402599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:26.402599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:26.402599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:26.402599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:26.402599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:26.402599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:26.402599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:26.402599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:26.403494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:26.404600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:26.404600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:26.404600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:26.404600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:26.404600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:26.404600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:26.404600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:26.404600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:26.404600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 344/711] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:26.411199 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51518219-77a8-4f70-968a-d569ad77d8ab None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:26.568521 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fee2b0-9852-4401-a3ec-f5db0c9b5219 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:26.590747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51518219-77a8-4f70-968a-d569ad77d8ab None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:26.612707 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ef2c8-e6df-4864-bc89-469cdaf92e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:26.613332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ef2c8-e6df-4864-bc89-469cdaf92e7b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:26.613600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ef2c8-e6df-4864-bc89-469cdaf92e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:26.613860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ef2c8-e6df-4864-bc89-469cdaf92e7b None admin] PATH_INFO: `/v3/services/network` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:26.614434 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ef2c8-e6df-4864-bc89-469cdaf92e7b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:26.616535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ef2c8-e6df-4864-bc89-469cdaf92e7b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c74ef2c8-e6df-4864-bc89-469cdaf92e7b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:26.629115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:26.630119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:26.630119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:26.630119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:26.630119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:26.630119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:26.630119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:26.630119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:26.630119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:26.630119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:26.630119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:26.630119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:26.630119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:26.630119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:26.630119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:26.630119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:26.630119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:26.631524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:26.631524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:26.631524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:26.631524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:26.631524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:26.631524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:26.631524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:26.631524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:26.631524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:26.631524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:26.631524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:26.631524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:26.631524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 07:54:26.631524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:26.631524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 368/712] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:26 2022] GET /identity/v3/services/network => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:26.632724 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ef2c8-e6df-4864-bc89-469cdaf92e7b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:26.635544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb1190-abe4-4599-9068-46e352cad0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:26.635544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb1190-abe4-4599-9068-46e352cad0b4 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:26.635544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb1190-abe4-4599-9068-46e352cad0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:26.635544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb1190-abe4-4599-9068-46e352cad0b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:26.636423 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb1190-abe4-4599-9068-46e352cad0b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:26.636733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb1190-abe4-4599-9068-46e352cad0b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:26.638177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb1190-abe4-4599-9068-46e352cad0b4 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:26.653510 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 345/713] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:26.654695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb1190-abe4-4599-9068-46e352cad0b4 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:26.659216 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:26.712710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:26.750579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ef2c8-e6df-4864-bc89-469cdaf92e7b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:26.786082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e394f-cebf-48e5-a034-2faf221acb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:26.786970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e394f-cebf-48e5-a034-2faf221acb03 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:26.787971 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e394f-cebf-48e5-a034-2faf221acb03 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:26.788354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e394f-cebf-48e5-a034-2faf221acb03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:26.790811 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb1190-abe4-4599-9068-46e352cad0b4 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:26.799454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e394f-cebf-48e5-a034-2faf221acb03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:26.801205 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e394f-cebf-48e5-a034-2faf221acb03 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:26.817238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1b6e394f-cebf-48e5-a034-2faf221acb03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:26.817238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:26.817238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:26.817238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:26.817238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:26.817238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:26.817238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:26.817238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:26.817238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:26.817238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:26.817238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:26.817238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:26.817238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:26.817238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:26.817238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:26.817238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:26.818527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:26.819869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:26.819869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:26.819869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:26.825815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 369/714] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:26.825815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e394f-cebf-48e5-a034-2faf221acb03 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:26.841951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29fe75-018c-44b7-98c0-870c5939a01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:26.843126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29fe75-018c-44b7-98c0-870c5939a01d None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:26.843556 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29fe75-018c-44b7-98c0-870c5939a01d None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:26.844145 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29fe75-018c-44b7-98c0-870c5939a01d None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:26.845255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29fe75-018c-44b7-98c0-870c5939a01d None admin] RBAC: Adding query filter params (name=network) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:26.845709 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29fe75-018c-44b7-98c0-870c5939a01d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:26.847277 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29fe75-018c-44b7-98c0-870c5939a01d None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:26.854409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 346/715] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Oct 6 07:54:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:26.855696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29fe75-018c-44b7-98c0-870c5939a01d None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:26.940288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:26.943879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e394f-cebf-48e5-a034-2faf221acb03 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:26.975760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29fe75-018c-44b7-98c0-870c5939a01d None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:26.977761 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:26.994017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8721504-8374-4b16-ab0d-efcdffc23948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:26.994872 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8721504-8374-4b16-ab0d-efcdffc23948 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:26.995496 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8721504-8374-4b16-ab0d-efcdffc23948 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:26.995927 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8721504-8374-4b16-ab0d-efcdffc23948 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:26.997074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8721504-8374-4b16-ab0d-efcdffc23948 None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=209795285384486183cda455fe316bc9) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:26.997534 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8721504-8374-4b16-ab0d-efcdffc23948 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:26.999483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8721504-8374-4b16-ab0d-efcdffc23948 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:27.010238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 370/716] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:26 2022] GET /identity/v3/role_assignments?user.id=209795285384486183cda455fe316bc9&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:27.011959 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8721504-8374-4b16-ab0d-efcdffc23948 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:27.016908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab4302-e4db-42cc-b418-f45e4efab914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:27.017716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab4302-e4db-42cc-b418-f45e4efab914 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:27.018087 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab4302-e4db-42cc-b418-f45e4efab914 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:27.018489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab4302-e4db-42cc-b418-f45e4efab914 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:27.019294 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab4302-e4db-42cc-b418-f45e4efab914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:27.019693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab4302-e4db-42cc-b418-f45e4efab914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:27.021276 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab4302-e4db-42cc-b418-f45e4efab914 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:27.035618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 347/717] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:27.044044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab4302-e4db-42cc-b418-f45e4efab914 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:27.155599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8721504-8374-4b16-ab0d-efcdffc23948 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:27.199704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f685d99-865c-4599-afe4-e299ec89b6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:27.200315 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f685d99-865c-4599-afe4-e299ec89b6cf None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:27.201084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f685d99-865c-4599-afe4-e299ec89b6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:27.201360 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f685d99-865c-4599-afe4-e299ec89b6cf None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:27.202014 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f685d99-865c-4599-afe4-e299ec89b6cf None admin] RBAC: Adding query filter params (type=network) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:27.202316 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f685d99-865c-4599-afe4-e299ec89b6cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:27.203728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f685d99-865c-4599-afe4-e299ec89b6cf None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:27.212481 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 371/718] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Oct 6 07:54:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:27.272012 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab4302-e4db-42cc-b418-f45e4efab914 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:27.309411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:27.318088 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23459a8a-684e-4943-90c8-6552651698d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:27.319473 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23459a8a-684e-4943-90c8-6552651698d7 None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:27.325370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23459a8a-684e-4943-90c8-6552651698d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:27.325370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23459a8a-684e-4943-90c8-6552651698d7 None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:27.325370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23459a8a-684e-4943-90c8-6552651698d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:27.325370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23459a8a-684e-4943-90c8-6552651698d7 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:27.387301 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d5c83-4ab4-47e8-ac7d-4a8caff260ea None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:27.387656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d5c83-4ab4-47e8-ac7d-4a8caff260ea None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:27.387841 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d5c83-4ab4-47e8-ac7d-4a8caff260ea None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:27.389513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 372/719] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:54:27.395233 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461eeef-707d-4e53-844a-76a2a0801dad None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:27.395513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461eeef-707d-4e53-844a-76a2a0801dad None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:27.395785 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461eeef-707d-4e53-844a-76a2a0801dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:27.417341 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6461eeef-707d-4e53-844a-76a2a0801dad None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:27.418565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6461eeef-707d-4e53-844a-76a2a0801dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:27.435837 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 348/720] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:54:27 2022] POST /identity/v3/users => generated 254 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:54:27.493252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:27.523123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 373/721] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:54:27.800007 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23459a8a-684e-4943-90c8-6552651698d7 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:27.909650 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23459a8a-684e-4943-90c8-6552651698d7 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:27.946563 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45665c02-823d-4b5f-aea9-f43060586bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:27.947348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45665c02-823d-4b5f-aea9-f43060586bce None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:27.947573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45665c02-823d-4b5f-aea9-f43060586bce None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:27.947839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45665c02-823d-4b5f-aea9-f43060586bce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:27.961578 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45665c02-823d-4b5f-aea9-f43060586bce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:27.967795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-45665c02-823d-4b5f-aea9-f43060586bce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:54:27.967795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:27.967795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:27.967795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:27.967795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:27.967795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:27.967795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:27.967795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:27.967795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:27.967795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:27.967795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:27.967795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:27.967795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:27.967795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:27.967795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:27.967795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:27.968737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:27.968737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:27.968737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:27.968737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:27.968737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:27.968737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:27.968737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:27.968737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:27.968737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:27.968737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:27.968737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:27.968737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:27.968737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:27.968737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:27.968737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:27.968737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:27.969568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:27.969568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:27.969568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:27.969568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:27.969568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:27.969568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:27.969568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:27.969568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:27.969568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:27.969568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:27.969568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:54:27.969568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:27.973711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 349/722] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:54:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:27.975716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461eeef-707d-4e53-844a-76a2a0801dad None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:28.109139 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461eeef-707d-4e53-844a-76a2a0801dad None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:28.158439 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ce9aa-88b2-47aa-bb7e-5f1d1846a0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:28.159144 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ce9aa-88b2-47aa-bb7e-5f1d1846a0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:28.159421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ce9aa-88b2-47aa-bb7e-5f1d1846a0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:28.159691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ce9aa-88b2-47aa-bb7e-5f1d1846a0d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:28.160356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ce9aa-88b2-47aa-bb7e-5f1d1846a0d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:28.160653 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ce9aa-88b2-47aa-bb7e-5f1d1846a0d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:28.162555 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ce9aa-88b2-47aa-bb7e-5f1d1846a0d7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:28.175135 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 374/723] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:54:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:28.181548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45665c02-823d-4b5f-aea9-f43060586bce None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:28.256160 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:28.334995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45665c02-823d-4b5f-aea9-f43060586bce None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:28.388279 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37acfda-a924-46a9-bef5-a0a4a7346d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:28.388843 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37acfda-a924-46a9-bef5-a0a4a7346d64 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:28.389124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37acfda-a924-46a9-bef5-a0a4a7346d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:28.389395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37acfda-a924-46a9-bef5-a0a4a7346d64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:28.401194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37acfda-a924-46a9-bef5-a0a4a7346d64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:28.402673 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37acfda-a924-46a9-bef5-a0a4a7346d64 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:28.417898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-c37acfda-a924-46a9-bef5-a0a4a7346d64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:28.417898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:28.417898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:28.417898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:28.417898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:28.417898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:28.417898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:28.417898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:28.417898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:28.417898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:28.417898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:28.417898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:28.417898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:28.417898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:28.417898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:28.417898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:28.418934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:28.419876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:28.419876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:28.419876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:28.419876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 350/724] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:28.426015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ce9aa-88b2-47aa-bb7e-5f1d1846a0d7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:28.472086 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53124204-e9ad-4d97-8a67-54221fb391f6 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:28.472422 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53124204-e9ad-4d97-8a67-54221fb391f6 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:28.472643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53124204-e9ad-4d97-8a67-54221fb391f6 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:28.474268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 351/725] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:28.479989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5536f6-4301-488d-bf1b-d17464501dc9 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:28.480274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5536f6-4301-488d-bf1b-d17464501dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:28.480823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5536f6-4301-488d-bf1b-d17464501dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:28.495406 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:28.514136 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-8e5536f6-4301-488d-bf1b-d17464501dc9 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:28.515379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-8e5536f6-4301-488d-bf1b-d17464501dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:28.607508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ce9aa-88b2-47aa-bb7e-5f1d1846a0d7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:28.645560 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 352/726] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:54:28.655419 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45682f-383c-455e-aa9d-97eb025ffb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:28.656021 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45682f-383c-455e-aa9d-97eb025ffb6c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:28.656296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45682f-383c-455e-aa9d-97eb025ffb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:28.656570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45682f-383c-455e-aa9d-97eb025ffb6c None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:28.657214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45682f-383c-455e-aa9d-97eb025ffb6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:28.657512 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45682f-383c-455e-aa9d-97eb025ffb6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:28.658867 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45682f-383c-455e-aa9d-97eb025ffb6c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:28.676161 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 375/727] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:28.682513 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5536f6-4301-488d-bf1b-d17464501dc9 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:28.715214 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:28.798862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5536f6-4301-488d-bf1b-d17464501dc9 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:28.834071 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7728005-af1e-4ccc-8aa6-0d17bc2b7a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:28.834786 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7728005-af1e-4ccc-8aa6-0d17bc2b7a54 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:28.835095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7728005-af1e-4ccc-8aa6-0d17bc2b7a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:28.835371 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7728005-af1e-4ccc-8aa6-0d17bc2b7a54 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:28.842904 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7728005-af1e-4ccc-8aa6-0d17bc2b7a54 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:28.844497 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7728005-af1e-4ccc-8aa6-0d17bc2b7a54 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:28.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-d7728005-af1e-4ccc-8aa6-0d17bc2b7a54 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 07:54:28.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:28.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:28.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:28.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:28.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:28.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:28.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:28.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:28.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:28.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:28.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:28.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:28.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:28.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:28.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:28.858146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:28.859134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:28.859134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:28.859134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:28.859134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:28.859134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:28.859134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:28.859134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:28.859134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:28.859134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 07:54:28.859134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:28.862962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 353/728] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:54:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:28.864297 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45682f-383c-455e-aa9d-97eb025ffb6c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:28.993588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45682f-383c-455e-aa9d-97eb025ffb6c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:28.999863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:29.043647 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea11e5-3a50-499d-950e-80ee518bef99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:29.044299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea11e5-3a50-499d-950e-80ee518bef99 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:29.044569 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea11e5-3a50-499d-950e-80ee518bef99 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:29.044835 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea11e5-3a50-499d-950e-80ee518bef99 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:29.045628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea11e5-3a50-499d-950e-80ee518bef99 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:29.045932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea11e5-3a50-499d-950e-80ee518bef99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:29.047386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea11e5-3a50-499d-950e-80ee518bef99 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:29.070274 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 376/729] 10.222.0.30 () {56 vars in 1135 bytes} [Thu Oct 6 07:54:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:29.075977 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7728005-af1e-4ccc-8aa6-0d17bc2b7a54 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:29.169747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea11e5-3a50-499d-950e-80ee518bef99 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:29.175677 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:29.188970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7728005-af1e-4ccc-8aa6-0d17bc2b7a54 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:29.226649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52f76c-5f2e-4976-b63c-87025fe32e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:29.227903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52f76c-5f2e-4976-b63c-87025fe32e86 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:29.228374 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52f76c-5f2e-4976-b63c-87025fe32e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:29.230835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52f76c-5f2e-4976-b63c-87025fe32e86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:29.340960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf52f76c-5f2e-4976-b63c-87025fe32e86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:29.342795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf52f76c-5f2e-4976-b63c-87025fe32e86 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:29.353767 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea11e5-3a50-499d-950e-80ee518bef99 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:29.391518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55589404-1e20-46f4-a53e-8e4d5d73cd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:29.392123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55589404-1e20-46f4-a53e-8e4d5d73cd74 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:29.392403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55589404-1e20-46f4-a53e-8e4d5d73cd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:29.392667 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55589404-1e20-46f4-a53e-8e4d5d73cd74 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:29.393314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55589404-1e20-46f4-a53e-8e4d5d73cd74 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:29.394665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55589404-1e20-46f4-a53e-8e4d5d73cd74 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:29.467337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:29.480183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 354/730] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 06 07:54:29.489610 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52f76c-5f2e-4976-b63c-87025fe32e86 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:29.519295 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 377/731] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Oct 6 07:54:29 2022] POST /identity/v3/services => generated 251 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 07:54:29.611556 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52f76c-5f2e-4976-b63c-87025fe32e86 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:29.658248 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983274b-c0a1-45a5-ab3c-76d8fe70feb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:29.658886 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983274b-c0a1-45a5-ab3c-76d8fe70feb7 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:29.659192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983274b-c0a1-45a5-ab3c-76d8fe70feb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:29.659464 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983274b-c0a1-45a5-ab3c-76d8fe70feb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:29.671973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983274b-c0a1-45a5-ab3c-76d8fe70feb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:29.673447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983274b-c0a1-45a5-ab3c-76d8fe70feb7 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:29.689751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-6983274b-c0a1-45a5-ab3c-76d8fe70feb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:29.689751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:29.689751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:29.689751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:29.689751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:29.689751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:29.689751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:29.689751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:29.689751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:29.689751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:29.689751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:29.689751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:29.689751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:29.689751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:29.689751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:29.689751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:29.690763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:29.691684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:29.691684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:29.691684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:29.691684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 355/732] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:29.699931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55589404-1e20-46f4-a53e-8e4d5d73cd74 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:29.721341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638a9b3-4be9-4a92-b5d7-14ccdb0a1054 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:29.721697 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638a9b3-4be9-4a92-b5d7-14ccdb0a1054 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:29.721881 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638a9b3-4be9-4a92-b5d7-14ccdb0a1054 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:29.730965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 356/733] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:54:29.730965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55a465-6f65-4bf8-9e94-acd0c21ce564 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:29.730965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55a465-6f65-4bf8-9e94-acd0c21ce564 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:29.730965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55a465-6f65-4bf8-9e94-acd0c21ce564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:29.747346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:29.780046 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-0f55a465-6f65-4bf8-9e94-acd0c21ce564 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:29.781296 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0f55a465-6f65-4bf8-9e94-acd0c21ce564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:29.832752 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55589404-1e20-46f4-a53e-8e4d5d73cd74 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:29.834327 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:29.881293 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7a73c-2c10-4d1a-9c9f-d0c19a84a77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:29.882709 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7a73c-2c10-4d1a-9c9f-d0c19a84a77c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:29.883354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7a73c-2c10-4d1a-9c9f-d0c19a84a77c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:29.883902 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7a73c-2c10-4d1a-9c9f-d0c19a84a77c None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:29.884879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7a73c-2c10-4d1a-9c9f-d0c19a84a77c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:29.885378 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7a73c-2c10-4d1a-9c9f-d0c19a84a77c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:29.887488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7a73c-2c10-4d1a-9c9f-d0c19a84a77c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:29.890667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 357/734] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 06 07:54:29.900963 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 378/735] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:29.907326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55a465-6f65-4bf8-9e94-acd0c21ce564 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:30.022416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55a465-6f65-4bf8-9e94-acd0c21ce564 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:30.066489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a08487-1c31-42fc-89f0-e3c66dbc6985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:30.067455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a08487-1c31-42fc-89f0-e3c66dbc6985 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:30.067908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a08487-1c31-42fc-89f0-e3c66dbc6985 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:30.069021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a08487-1c31-42fc-89f0-e3c66dbc6985 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:30.081696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a08487-1c31-42fc-89f0-e3c66dbc6985 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:30.083896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a08487-1c31-42fc-89f0-e3c66dbc6985 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:30.097919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-12a08487-1c31-42fc-89f0-e3c66dbc6985 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:30.097919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:30.097919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:30.097919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:30.097919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:30.097919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:30.097919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:30.097919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:30.097919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:30.097919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:30.097919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:30.097919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:30.097919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:30.097919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:30.097919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:30.098804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:30.099862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:30.099862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:30.099862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:30.099862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:30.099862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:30.099862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:30.099862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:30.099862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:30.102026 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 358/736] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:30.107475 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7a73c-2c10-4d1a-9c9f-d0c19a84a77c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:30.236251 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:30.256413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fccf6-0ee6-4fe2-9f00-cba5cc815c6e None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:30.257180 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fccf6-0ee6-4fe2-9f00-cba5cc815c6e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:30.257865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fccf6-0ee6-4fe2-9f00-cba5cc815c6e None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:30.259260 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7a73c-2c10-4d1a-9c9f-d0c19a84a77c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:30.259954 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 359/737] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:30.261179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fccf6-0ee6-4fe2-9f00-cba5cc815c6e None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:30.295012 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:30.300354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80bd77-ab9d-4494-b88b-37b83ce01fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:30.301051 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80bd77-ab9d-4494-b88b-37b83ce01fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:30.301431 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80bd77-ab9d-4494-b88b-37b83ce01fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:30.301721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80bd77-ab9d-4494-b88b-37b83ce01fc5 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:30.303044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80bd77-ab9d-4494-b88b-37b83ce01fc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:30.303044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80bd77-ab9d-4494-b88b-37b83ce01fc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:30.307848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80bd77-ab9d-4494-b88b-37b83ce01fc5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:30.319197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 379/738] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:30.321182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd95ba-812a-4f23-9e77-8f7130740f2c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:30.321524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd95ba-812a-4f23-9e77-8f7130740f2c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:30.321856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd95ba-812a-4f23-9e77-8f7130740f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:30.343312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-32dd95ba-812a-4f23-9e77-8f7130740f2c None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:30.344712 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-32dd95ba-812a-4f23-9e77-8f7130740f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:30.385315 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fccf6-0ee6-4fe2-9f00-cba5cc815c6e None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:30.420446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85246b8-a959-416f-8a49-1abcbaa5b15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:30.420990 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85246b8-a959-416f-8a49-1abcbaa5b15d None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:30.421269 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85246b8-a959-416f-8a49-1abcbaa5b15d None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:30.421540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85246b8-a959-416f-8a49-1abcbaa5b15d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:30.422672 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 380/739] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:30.423883 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd95ba-812a-4f23-9e77-8f7130740f2c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:30.438298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85246b8-a959-416f-8a49-1abcbaa5b15d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:30.444404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-d85246b8-a959-416f-8a49-1abcbaa5b15d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:54:30.444404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:30.444404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:30.444404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:30.444404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:30.444404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:30.444404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:30.444404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:30.444404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:30.444404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:30.444404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:30.444404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:30.444404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:30.444404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:30.444404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:30.444404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:30.445341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:30.445341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:30.445341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:30.445341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:30.445341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:30.445341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:30.445341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:30.445341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:30.445341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:30.445341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:30.445341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:30.445341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:30.445341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:30.445341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:30.445341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:30.445341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:30.446206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:30.446206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:30.446206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:30.446206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:30.446206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:30.446206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:30.446206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:30.446206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:30.446206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:30.446206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:30.446206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:54:30.446206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:30.448946 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 360/740] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:54:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:30.450579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30617af2-2803-4b57-ab8d-d9d6640eccb3 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:30.451213 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30617af2-2803-4b57-ab8d-d9d6640eccb3 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:30.451685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30617af2-2803-4b57-ab8d-d9d6640eccb3 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:30.453602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 361/741] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:30.454574 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30617af2-2803-4b57-ab8d-d9d6640eccb3 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:30.507504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:30.548150 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd95ba-812a-4f23-9e77-8f7130740f2c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:30.559005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:30.584690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30617af2-2803-4b57-ab8d-d9d6640eccb3 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:30.593714 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c83f3b-b87c-4ac7-ad22-78628255838b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:30.595067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c83f3b-b87c-4ac7-ad22-78628255838b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:30.595536 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c83f3b-b87c-4ac7-ad22-78628255838b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:30.595966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c83f3b-b87c-4ac7-ad22-78628255838b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:30.596857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c83f3b-b87c-4ac7-ad22-78628255838b None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=3fa1c430c9f7416eaf1d17300b827e05) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:30.597752 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c83f3b-b87c-4ac7-ad22-78628255838b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:30.600848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c83f3b-b87c-4ac7-ad22-78628255838b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:30.615573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 381/742] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:30 2022] GET /identity/v3/role_assignments?user.id=3fa1c430c9f7416eaf1d17300b827e05&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:30.615573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73077ec-8da0-4830-8813-07c0cf198f63 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:30.615573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73077ec-8da0-4830-8813-07c0cf198f63 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:30.615573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73077ec-8da0-4830-8813-07c0cf198f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:30.657800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f73077ec-8da0-4830-8813-07c0cf198f63 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:30.659065 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f73077ec-8da0-4830-8813-07c0cf198f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:30.667402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d0a5b-fcbe-45f3-a32e-eedacd628195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:30.668088 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d0a5b-fcbe-45f3-a32e-eedacd628195 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:30.668366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d0a5b-fcbe-45f3-a32e-eedacd628195 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:30.668636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d0a5b-fcbe-45f3-a32e-eedacd628195 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:30.675456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d0a5b-fcbe-45f3-a32e-eedacd628195 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:30.675757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d0a5b-fcbe-45f3-a32e-eedacd628195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:30.677198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d0a5b-fcbe-45f3-a32e-eedacd628195 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:30.697096 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 362/743] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:54:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:30.702537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d0a5b-fcbe-45f3-a32e-eedacd628195 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:30.780044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 382/744] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 06 07:54:30.780941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73077ec-8da0-4830-8813-07c0cf198f63 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:30.846718 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d0a5b-fcbe-45f3-a32e-eedacd628195 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:30.873221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:30.900606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27b858-64b3-46a8-9d42-a1255575f615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:30.901287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27b858-64b3-46a8-9d42-a1255575f615 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:30.901625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27b858-64b3-46a8-9d42-a1255575f615 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:30.901957 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27b858-64b3-46a8-9d42-a1255575f615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:30.924098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f27b858-64b3-46a8-9d42-a1255575f615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:30.925774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f27b858-64b3-46a8-9d42-a1255575f615 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:30.939838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73077ec-8da0-4830-8813-07c0cf198f63 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:30.943258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:30.960207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-3f27b858-64b3-46a8-9d42-a1255575f615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:30.960207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:30.960207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:30.960207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:30.960207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:30.960207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:30.960207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:30.960207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:30.960207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:30.960207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:30.960207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:30.960207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:30.960207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:30.960207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:30.960207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:30.960207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:30.961225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:30.962684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:30.962684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:30.962684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:30.962684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 363/745] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:30.968584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27b858-64b3-46a8-9d42-a1255575f615 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:31.015662 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadb3b0-75b2-414d-af1f-36075c2d964b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:31.016268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadb3b0-75b2-414d-af1f-36075c2d964b None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:31.016539 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadb3b0-75b2-414d-af1f-36075c2d964b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:31.016803 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadb3b0-75b2-414d-af1f-36075c2d964b None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:31.017459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aadb3b0-75b2-414d-af1f-36075c2d964b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:31.020257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aadb3b0-75b2-414d-af1f-36075c2d964b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:31.092023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27b858-64b3-46a8-9d42-a1255575f615 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:31.114207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 383/746] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Oct 6 07:54:30 2022] POST /identity/v3/services => generated 240 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:54:31.115789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadb3b0-75b2-414d-af1f-36075c2d964b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:31.145877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40de16c-1a65-49a8-bd0c-3ae1994012af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:31.147416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40de16c-1a65-49a8-bd0c-3ae1994012af None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:31.147704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40de16c-1a65-49a8-bd0c-3ae1994012af None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:31.147970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40de16c-1a65-49a8-bd0c-3ae1994012af None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:31.148632 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40de16c-1a65-49a8-bd0c-3ae1994012af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:31.148938 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40de16c-1a65-49a8-bd0c-3ae1994012af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:31.150516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40de16c-1a65-49a8-bd0c-3ae1994012af None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:31.167180 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 364/747] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:31.171118 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40de16c-1a65-49a8-bd0c-3ae1994012af None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:31.232851 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadb3b0-75b2-414d-af1f-36075c2d964b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:31.247794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:31.270427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e0949-2169-4664-892a-152518de1bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:31.271649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e0949-2169-4664-892a-152518de1bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:31.271906 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e0949-2169-4664-892a-152518de1bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:31.272261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e0949-2169-4664-892a-152518de1bb6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:31.277366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:31.298888 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e0949-2169-4664-892a-152518de1bb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:31.304509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-f14e0949-2169-4664-892a-152518de1bb6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:31.304509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:31.304509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:31.304509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:31.304509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:31.304509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:31.304509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:31.304509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:31.304509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:31.304509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:31.304509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:31.304509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:31.304509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:31.304509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:31.304509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:31.304509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:31.306125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:31.306125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:31.306125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:31.306125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:31.306125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:31.306125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:31.306125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:31.306125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:31.306125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:31.306125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:31.306125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:31.306125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:31.306125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:31.306125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:31.306125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:31.306125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:31.307634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:31.307634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:31.307634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:31.307634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:31.307634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:31.307634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:31.307634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:31.307634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:31.307634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:31.307634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:31.307634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:31.307634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:31.307634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 384/748] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:31.323066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e0949-2169-4664-892a-152518de1bb6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:31.351072 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40de16c-1a65-49a8-bd0c-3ae1994012af None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:31.408537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f2cea-4b55-4e7e-b9bc-baed1b9a65e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:31.409178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f2cea-4b55-4e7e-b9bc-baed1b9a65e2 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:31.409460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f2cea-4b55-4e7e-b9bc-baed1b9a65e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:31.409737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f2cea-4b55-4e7e-b9bc-baed1b9a65e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:31.421864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f2cea-4b55-4e7e-b9bc-baed1b9a65e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:31.423890 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f2cea-4b55-4e7e-b9bc-baed1b9a65e2 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:31.440824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-732f2cea-4b55-4e7e-b9bc-baed1b9a65e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:54:31.440824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:31.440824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:31.440824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:31.440824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:31.440824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:31.440824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:31.440824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:31.440824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:31.440824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:31.440824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:31.440824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:31.440824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:31.440824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:31.440824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:31.440824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:31.441828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:31.442774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:31.442774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:31.442774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:31.442774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:31.442774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:31.442774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:31.442774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:31.442774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:31.442774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:54:31.442774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:31.444169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 365/749] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Oct 6 07:54:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:31.449865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f2cea-4b55-4e7e-b9bc-baed1b9a65e2 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:31.468269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e0949-2169-4664-892a-152518de1bb6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:31.508797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa43b31-2737-4008-82df-8c37f8c3f33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:31.509408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa43b31-2737-4008-82df-8c37f8c3f33a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:31.510165 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa43b31-2737-4008-82df-8c37f8c3f33a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:31.510432 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa43b31-2737-4008-82df-8c37f8c3f33a None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:31.511158 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa43b31-2737-4008-82df-8c37f8c3f33a None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:31.511457 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa43b31-2737-4008-82df-8c37f8c3f33a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:31.513198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa43b31-2737-4008-82df-8c37f8c3f33a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:31.524377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 385/750] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:31.529895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa43b31-2737-4008-82df-8c37f8c3f33a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:31.590489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f2cea-4b55-4e7e-b9bc-baed1b9a65e2 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:31.595355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:31.611617 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:31.652717 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d37faf-1289-460c-9f39-869f51be891f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:31.653335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d37faf-1289-460c-9f39-869f51be891f None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:31.653617 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d37faf-1289-460c-9f39-869f51be891f None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:31.653891 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d37faf-1289-460c-9f39-869f51be891f None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:31.654620 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d37faf-1289-460c-9f39-869f51be891f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:31.654924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d37faf-1289-460c-9f39-869f51be891f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:31.656383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d37faf-1289-460c-9f39-869f51be891f None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:31.677112 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 366/751] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Oct 6 07:54:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:31.682860 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d37faf-1289-460c-9f39-869f51be891f None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:31.712306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa43b31-2737-4008-82df-8c37f8c3f33a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:31.806090 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d37faf-1289-460c-9f39-869f51be891f None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:31.824228 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8ecef-3298-42d6-8e0b-740b737b54f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:31.824874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8ecef-3298-42d6-8e0b-740b737b54f9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:31.825150 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8ecef-3298-42d6-8e0b-740b737b54f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:31.825422 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8ecef-3298-42d6-8e0b-740b737b54f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:31.842772 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4074e8b-c095-4f12-8d8e-5ec66bf6b07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:31.843865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4074e8b-c095-4f12-8d8e-5ec66bf6b07d None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:31.844152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4074e8b-c095-4f12-8d8e-5ec66bf6b07d None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:31.844423 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4074e8b-c095-4f12-8d8e-5ec66bf6b07d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:31.852684 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8ecef-3298-42d6-8e0b-740b737b54f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:31.854139 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8ecef-3298-42d6-8e0b-740b737b54f9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:31.856122 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4074e8b-c095-4f12-8d8e-5ec66bf6b07d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:31.858554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4074e8b-c095-4f12-8d8e-5ec66bf6b07d None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:31.874680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-d4074e8b-c095-4f12-8d8e-5ec66bf6b07d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:31.874680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:31.874680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:31.874680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:31.874680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:31.874680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:31.874680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:31.874680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:31.874680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:31.874680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:31.874680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:31.874680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:31.874680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:31.874680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:31.874680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:31.874680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:31.875666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:31.876554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:31.876554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:31.876554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:31.876554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 367/752] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:31.882011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-24f8ecef-3298-42d6-8e0b-740b737b54f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:31.882011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:31.882011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:31.882011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:31.882011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:31.882011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:31.882011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:31.882011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:31.882011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:31.882011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:31.882011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:31.882011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:31.882011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:31.882011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:31.882011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:31.882011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:31.882957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:31.883860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:31.883860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:31.883860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:31.886455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4074e8b-c095-4f12-8d8e-5ec66bf6b07d None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:31.887043 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 386/753] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:31.888883 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8ecef-3298-42d6-8e0b-740b737b54f9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:31.916372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:32.031704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4074e8b-c095-4f12-8d8e-5ec66bf6b07d None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:32.107589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:32.111114 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8ecef-3298-42d6-8e0b-740b737b54f9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:32.112881 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96094dcb-8306-4937-9f86-877acf8d9f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:32.114031 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96094dcb-8306-4937-9f86-877acf8d9f58 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:32.114512 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96094dcb-8306-4937-9f86-877acf8d9f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:32.115653 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96094dcb-8306-4937-9f86-877acf8d9f58 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:32.123832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96094dcb-8306-4937-9f86-877acf8d9f58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:32.123832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96094dcb-8306-4937-9f86-877acf8d9f58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:32.123832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96094dcb-8306-4937-9f86-877acf8d9f58 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:32.140272 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 368/754] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:32.145714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96094dcb-8306-4937-9f86-877acf8d9f58 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:32.165702 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af4fc4-6b23-4101-9dd3-d3b76b5ebcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:32.166635 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af4fc4-6b23-4101-9dd3-d3b76b5ebcec None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:32.167136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af4fc4-6b23-4101-9dd3-d3b76b5ebcec None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:32.167604 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af4fc4-6b23-4101-9dd3-d3b76b5ebcec None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:32.171201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af4fc4-6b23-4101-9dd3-d3b76b5ebcec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:32.171724 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af4fc4-6b23-4101-9dd3-d3b76b5ebcec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:32.173321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af4fc4-6b23-4101-9dd3-d3b76b5ebcec None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:32.198481 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 387/755] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:32.202859 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af4fc4-6b23-4101-9dd3-d3b76b5ebcec None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:32.295219 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:32.314336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96094dcb-8306-4937-9f86-877acf8d9f58 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:32.332424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af4fc4-6b23-4101-9dd3-d3b76b5ebcec None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:32.356763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e1121-e768-4ce5-a7a7-c654a925ce19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:32.357820 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1121-e768-4ce5-a7a7-c654a925ce19 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:32.358292 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1121-e768-4ce5-a7a7-c654a925ce19 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:32.358789 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1121-e768-4ce5-a7a7-c654a925ce19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:32.367511 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e1121-e768-4ce5-a7a7-c654a925ce19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:32.368012 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34670e50-56b2-41a4-9493-55fe668d0551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:32.368804 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34670e50-56b2-41a4-9493-55fe668d0551 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:32.369254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34670e50-56b2-41a4-9493-55fe668d0551 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:32.369481 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e1121-e768-4ce5-a7a7-c654a925ce19 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:32.369825 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34670e50-56b2-41a4-9493-55fe668d0551 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:32.377946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34670e50-56b2-41a4-9493-55fe668d0551 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:32.379605 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34670e50-56b2-41a4-9493-55fe668d0551 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:32.384906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-d35e1121-e768-4ce5-a7a7-c654a925ce19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:32.384906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:32.384906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:32.384906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:32.384906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:32.384906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:32.384906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:32.384906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:32.384906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:32.384906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:32.384906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:32.384906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:32.384906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:32.384906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:32.384906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:32.385880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:32.386794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:32.386794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:32.386794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:32.386794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:32.386794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:32.386794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:32.386794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:32.386794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:32.386794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 369/756] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:32.388118 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ac989-5497-4501-aa03-33d3fe3dc6bb None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:32.388514 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ac989-5497-4501-aa03-33d3fe3dc6bb None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:32.388848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ac989-5497-4501-aa03-33d3fe3dc6bb None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:32.390626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 370/757] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:32.395247 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ac989-5497-4501-aa03-33d3fe3dc6bb None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:32.397411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-34670e50-56b2-41a4-9493-55fe668d0551 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 07:54:32.397411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:32.397411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:32.397411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:32.397411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:32.397411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:32.397411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:32.397411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:32.397411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:32.397411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:32.397411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:32.397411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:32.397411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:32.397411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:32.397411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:32.397411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:32.398541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:32.399734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:32.399734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:32.399734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:32.399734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:32.399734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:32.399734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:32.399734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:32.399734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:32.399734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 07:54:32.399734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:32.401495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 388/758] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Oct 6 07:54:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:32.402773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbadd0-115a-4f94-8020-491786aa3069 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:32.403175 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbadd0-115a-4f94-8020-491786aa3069 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:32.403540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbadd0-115a-4f94-8020-491786aa3069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:32.423704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-00dbadd0-115a-4f94-8020-491786aa3069 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:32.425061 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-00dbadd0-115a-4f94-8020-491786aa3069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:32.441961 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:32.507381 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 389/759] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:54:32.508378 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbadd0-115a-4f94-8020-491786aa3069 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:32.535152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ac989-5497-4501-aa03-33d3fe3dc6bb None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:32.580873 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33156ff9-7017-49aa-84e9-0e8bcc8d2761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:32.581757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33156ff9-7017-49aa-84e9-0e8bcc8d2761 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:32.582587 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33156ff9-7017-49aa-84e9-0e8bcc8d2761 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:32.583343 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33156ff9-7017-49aa-84e9-0e8bcc8d2761 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:32.585189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33156ff9-7017-49aa-84e9-0e8bcc8d2761 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:32.585638 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33156ff9-7017-49aa-84e9-0e8bcc8d2761 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:32.587272 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33156ff9-7017-49aa-84e9-0e8bcc8d2761 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:32.601063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 371/760] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:32.606976 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33156ff9-7017-49aa-84e9-0e8bcc8d2761 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:32.611599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:32.749473 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbadd0-115a-4f94-8020-491786aa3069 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:32.799721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea6395-4c84-4ee2-b1f9-316adacd6e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:32.800921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea6395-4c84-4ee2-b1f9-316adacd6e52 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:32.801550 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea6395-4c84-4ee2-b1f9-316adacd6e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:32.802119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea6395-4c84-4ee2-b1f9-316adacd6e52 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:32.803243 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea6395-4c84-4ee2-b1f9-316adacd6e52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:32.804208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea6395-4c84-4ee2-b1f9-316adacd6e52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:32.806053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea6395-4c84-4ee2-b1f9-316adacd6e52 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:32.814588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33156ff9-7017-49aa-84e9-0e8bcc8d2761 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:32.827079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 390/761] 10.222.0.30 () {56 vars in 1149 bytes} [Thu Oct 6 07:54:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:32.833497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea6395-4c84-4ee2-b1f9-316adacd6e52 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:32.845031 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:32.868711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41387b7f-ff8d-4b04-8ebe-557714799db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:32.870801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41387b7f-ff8d-4b04-8ebe-557714799db7 None admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:32.871497 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41387b7f-ff8d-4b04-8ebe-557714799db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:32.871978 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41387b7f-ff8d-4b04-8ebe-557714799db7 None admin] PATH_INFO: `/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/209795285384486183cda455fe316bc9/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:32.874549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41387b7f-ff8d-4b04-8ebe-557714799db7 None admin] RBAC: Authorizing `identity:create_grant(project_id=399f0095cc8b4c64893ccc75af13c52d, user_id=209795285384486183cda455fe316bc9, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:32.877387 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41387b7f-ff8d-4b04-8ebe-557714799db7 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:32.945756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea6395-4c84-4ee2-b1f9-316adacd6e52 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:32.957822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 372/762] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:54:32 2022] PUT /identity/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/209795285384486183cda455fe316bc9/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 07:54:32.961026 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41387b7f-ff8d-4b04-8ebe-557714799db7 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:33.020190 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13911c6-32c3-4caa-bbc7-88da3f13ae33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:33.020844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13911c6-32c3-4caa-bbc7-88da3f13ae33 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:33.021156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13911c6-32c3-4caa-bbc7-88da3f13ae33 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:33.021640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13911c6-32c3-4caa-bbc7-88da3f13ae33 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:33.022241 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13911c6-32c3-4caa-bbc7-88da3f13ae33 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:33.024838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13911c6-32c3-4caa-bbc7-88da3f13ae33 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a13911c6-32c3-4caa-bbc7-88da3f13ae33 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:33.046278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:33.050372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:33.050372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:33.050372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:33.050372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:33.050372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:33.050372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:33.050372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:33.050372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:33.050372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:33.050372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:33.050372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:33.050372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:33.050372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:33.050372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:33.050372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:33.050372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:33.051289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:33.051289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:33.051289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:33.051289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:33.051289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:33.051289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:33.051289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:33.051289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:33.051289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:33.051289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:33.051289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:33.051289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:33.051289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 07:54:33.051289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:33.061113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 391/763] 10.222.0.30 () {56 vars in 1129 bytes} [Thu Oct 6 07:54:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:33.065937 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13911c6-32c3-4caa-bbc7-88da3f13ae33 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:33.134436 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:33.175363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:33.208252 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41387b7f-ff8d-4b04-8ebe-557714799db7 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:33.217690 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13911c6-32c3-4caa-bbc7-88da3f13ae33 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:33.245305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41315234-a4d2-4d40-a680-f4a85d358004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:33.246335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41315234-a4d2-4d40-a680-f4a85d358004 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:33.246610 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41315234-a4d2-4d40-a680-f4a85d358004 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:33.247142 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41315234-a4d2-4d40-a680-f4a85d358004 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:33.259027 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a35e2-520e-4cc0-8a3d-0729dfc55bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:33.259643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a35e2-520e-4cc0-8a3d-0729dfc55bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:33.259930 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a35e2-520e-4cc0-8a3d-0729dfc55bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:33.260207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a35e2-520e-4cc0-8a3d-0729dfc55bd2 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:33.260845 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a35e2-520e-4cc0-8a3d-0729dfc55bd2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:33.261149 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a35e2-520e-4cc0-8a3d-0729dfc55bd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:33.262533 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a35e2-520e-4cc0-8a3d-0729dfc55bd2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:33.267814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 392/764] 10.222.0.30 () {56 vars in 1124 bytes} [Thu Oct 6 07:54:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:33.269017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bb90b-76ba-45d8-b8ea-4c474a8c3bd7 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:33.269301 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bb90b-76ba-45d8-b8ea-4c474a8c3bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:33.269578 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bb90b-76ba-45d8-b8ea-4c474a8c3bd7 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:33.271232 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 393/765] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:33.275825 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bb90b-76ba-45d8-b8ea-4c474a8c3bd7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:33.333053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41315234-a4d2-4d40-a680-f4a85d358004 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:33.336235 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41315234-a4d2-4d40-a680-f4a85d358004 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:33.387066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bb90b-76ba-45d8-b8ea-4c474a8c3bd7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:33.418295 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:33.426258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0f235-cbca-4370-8ced-f3221612cf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:33.427042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f235-cbca-4370-8ced-f3221612cf18 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:33.436549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f235-cbca-4370-8ced-f3221612cf18 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:33.436549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f235-cbca-4370-8ced-f3221612cf18 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:33.436549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0f235-cbca-4370-8ced-f3221612cf18 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:33.436549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0f235-cbca-4370-8ced-f3221612cf18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:33.436549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0f235-cbca-4370-8ced-f3221612cf18 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:33.436549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:33.440020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 394/766] 10.222.0.30 () {56 vars in 1124 bytes} [Thu Oct 6 07:54:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:33.441457 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630935c-3fe3-44cb-97e8-c3be5a3f46b5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:33.441947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630935c-3fe3-44cb-97e8-c3be5a3f46b5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:33.442447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630935c-3fe3-44cb-97e8-c3be5a3f46b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:33.463606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2630935c-3fe3-44cb-97e8-c3be5a3f46b5 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:33.467202 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2630935c-3fe3-44cb-97e8-c3be5a3f46b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:33.478854 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 373/767] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 07:54:33.479941 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41315234-a4d2-4d40-a680-f4a85d358004 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:33.563306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 395/768] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:54:33.563306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630935c-3fe3-44cb-97e8-c3be5a3f46b5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:33.647380 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41315234-a4d2-4d40-a680-f4a85d358004 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:33.700214 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0302d9-0f8a-4cf7-a62b-af81a70276cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:33.701316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0302d9-0f8a-4cf7-a62b-af81a70276cc None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:33.701801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0302d9-0f8a-4cf7-a62b-af81a70276cc None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:33.702305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0302d9-0f8a-4cf7-a62b-af81a70276cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:33.703317 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0302d9-0f8a-4cf7-a62b-af81a70276cc None admin] RBAC: Adding query filter params (interface=public, service_id=9ff078853d12480180a25144f908ecd4, region_id=RegionOne) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:33.703834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0302d9-0f8a-4cf7-a62b-af81a70276cc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:33.705694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0302d9-0f8a-4cf7-a62b-af81a70276cc None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:33.714211 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 374/769] 10.222.0.30 () {56 vars in 1251 bytes} [Thu Oct 6 07:54:33 2022] GET /identity/v3/endpoints?service_id=9ff078853d12480180a25144f908ecd4&interface=public®ion_id=RegionOne => generated 194 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:33.720089 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0302d9-0f8a-4cf7-a62b-af81a70276cc None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:33.723828 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630935c-3fe3-44cb-97e8-c3be5a3f46b5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:33.769274 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:33.777579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:33.784275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135fff4-3aa5-4b36-8b38-2456a53bbb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:33.793199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135fff4-3aa5-4b36-8b38-2456a53bbb35 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:33.793199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135fff4-3aa5-4b36-8b38-2456a53bbb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:33.793199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135fff4-3aa5-4b36-8b38-2456a53bbb35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:33.808554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135fff4-3aa5-4b36-8b38-2456a53bbb35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:33.810311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135fff4-3aa5-4b36-8b38-2456a53bbb35 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:33.840195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3135fff4-3aa5-4b36-8b38-2456a53bbb35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:33.840195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:33.840195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:33.840195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:33.840195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:33.840195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:33.840195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:33.840195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:33.840195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:33.840195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:33.840195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:33.840195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:33.840195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:33.840195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:33.840195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:33.840195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:33.841445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:33.842583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:33.842583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:33.842583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:33.846419 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 396/770] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:33.851628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135fff4-3aa5-4b36-8b38-2456a53bbb35 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:33.925330 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0302d9-0f8a-4cf7-a62b-af81a70276cc None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:33.964505 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ffd32-3ba0-4e9b-a6e0-aa16a428450c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:33.965579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ffd32-3ba0-4e9b-a6e0-aa16a428450c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:33.965865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ffd32-3ba0-4e9b-a6e0-aa16a428450c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:33.966479 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ffd32-3ba0-4e9b-a6e0-aa16a428450c None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:33.967156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ffd32-3ba0-4e9b-a6e0-aa16a428450c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:33.968577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ffd32-3ba0-4e9b-a6e0-aa16a428450c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:33.974877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 375/771] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Oct 6 07:54:33 2022] GET /identity/v3/services => generated 772 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:33.976650 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a5cd0-7c8f-411b-a47a-52ac5bf01242 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:33.976936 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a5cd0-7c8f-411b-a47a-52ac5bf01242 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:33.977207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a5cd0-7c8f-411b-a47a-52ac5bf01242 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:33.982761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 376/772] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:33.982761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a5cd0-7c8f-411b-a47a-52ac5bf01242 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:34.007826 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135fff4-3aa5-4b36-8b38-2456a53bbb35 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:34.052331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e1c13-3e4b-4ee1-aed3-1820ec05f39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:34.054225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e1c13-3e4b-4ee1-aed3-1820ec05f39c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:34.054561 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e1c13-3e4b-4ee1-aed3-1820ec05f39c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:34.055141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e1c13-3e4b-4ee1-aed3-1820ec05f39c None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:34.055681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e1c13-3e4b-4ee1-aed3-1820ec05f39c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:34.056054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e1c13-3e4b-4ee1-aed3-1820ec05f39c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:34.057549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e1c13-3e4b-4ee1-aed3-1820ec05f39c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:34.076732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 397/773] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:34.076732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a2ef3-e75d-46e2-acd1-321aabcfaa47 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:34.076732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a2ef3-e75d-46e2-acd1-321aabcfaa47 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:34.076732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a2ef3-e75d-46e2-acd1-321aabcfaa47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:34.097785 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f49a2ef3-e75d-46e2-acd1-321aabcfaa47 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:34.100599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f49a2ef3-e75d-46e2-acd1-321aabcfaa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:34.114196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:34.126196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a5cd0-7c8f-411b-a47a-52ac5bf01242 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:34.156723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:34.171704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1c90c-c9f7-40a5-b6aa-e899d4628ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:34.172329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1c90c-c9f7-40a5-b6aa-e899d4628ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:34.172601 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1c90c-c9f7-40a5-b6aa-e899d4628ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:34.172868 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1c90c-c9f7-40a5-b6aa-e899d4628ff8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:34.185181 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc1c90c-c9f7-40a5-b6aa-e899d4628ff8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:34.189907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-5bc1c90c-c9f7-40a5-b6aa-e899d4628ff8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:54:34.189907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:34.189907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:34.189907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:34.189907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:34.189907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:34.189907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:34.189907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:34.189907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:34.189907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:34.189907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:34.189907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:34.189907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:34.189907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:34.189907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:34.189907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:34.191168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:34.191168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:34.191168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:34.191168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:34.191168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:34.191168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:34.191168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:34.191168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:34.191168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:34.191168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:34.191168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:34.191168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:34.191168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:34.191168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:34.191168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:34.191168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:34.192038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:34.192038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:34.192038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:34.192038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:34.192038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:34.192038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:34.192038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:34.192038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:34.192038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:34.192038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:34.192038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:54:34.192038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:34.192038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 377/774] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:54:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:34.192921 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1c90c-c9f7-40a5-b6aa-e899d4628ff8 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:34.203081 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 398/775] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:54:34.203997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a2ef3-e75d-46e2-acd1-321aabcfaa47 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:34.330980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a2ef3-e75d-46e2-acd1-321aabcfaa47 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:34.359180 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1c90c-c9f7-40a5-b6aa-e899d4628ff8 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:34.374208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845376aa-fa51-4090-a1a7-0ed0bb4f992a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:34.374819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845376aa-fa51-4090-a1a7-0ed0bb4f992a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:34.375133 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845376aa-fa51-4090-a1a7-0ed0bb4f992a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:34.375403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845376aa-fa51-4090-a1a7-0ed0bb4f992a None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:34.376068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845376aa-fa51-4090-a1a7-0ed0bb4f992a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:34.376365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845376aa-fa51-4090-a1a7-0ed0bb4f992a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:34.378318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845376aa-fa51-4090-a1a7-0ed0bb4f992a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:34.392323 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 399/776] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:54:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:34.402194 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845376aa-fa51-4090-a1a7-0ed0bb4f992a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:34.491657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cea54-56c4-4a96-958f-8e8f78149c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:34.492303 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cea54-56c4-4a96-958f-8e8f78149c5e None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:34.492579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cea54-56c4-4a96-958f-8e8f78149c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:34.492849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cea54-56c4-4a96-958f-8e8f78149c5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:34.493409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:34.523660 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cea54-56c4-4a96-958f-8e8f78149c5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:34.525153 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cea54-56c4-4a96-958f-8e8f78149c5e None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:34.562867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-c23cea54-56c4-4a96-958f-8e8f78149c5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:34.562867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:34.562867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:34.562867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:34.562867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:34.562867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:34.562867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:34.562867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:34.562867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:34.562867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:34.562867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:34.562867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:34.562867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:34.562867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:34.562867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:34.563851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:34.564747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:34.564747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:34.564747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:34.564747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:34.564747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:34.564747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:34.564747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:34.564747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:34.574298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 378/777] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:34.575568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cea54-56c4-4a96-958f-8e8f78149c5e None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:34.597912 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:34.665041 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845376aa-fa51-4090-a1a7-0ed0bb4f992a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:34.729350 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f7a90-d52f-4733-a8e9-143c5e7da173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:34.729994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f7a90-d52f-4733-a8e9-143c5e7da173 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:34.730270 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f7a90-d52f-4733-a8e9-143c5e7da173 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:34.730533 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f7a90-d52f-4733-a8e9-143c5e7da173 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:34.757039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cea54-56c4-4a96-958f-8e8f78149c5e None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:34.763319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f7a90-d52f-4733-a8e9-143c5e7da173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:34.764780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f7a90-d52f-4733-a8e9-143c5e7da173 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:34.814755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e28f7a90-d52f-4733-a8e9-143c5e7da173 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:34.814755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:34.814755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:34.814755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:34.814755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:34.814755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:34.814755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:34.814755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:34.814755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:34.814755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:34.814755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:34.814755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:34.814755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:34.814755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:34.814755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:34.814755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:34.816234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:34.817233 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:34.817233 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:34.817233 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:34.820354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 400/778] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:34.820912 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b879f72-f1b0-4add-8642-d5bcaf9aa2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:34.821862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b879f72-f1b0-4add-8642-d5bcaf9aa2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:34.822341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b879f72-f1b0-4add-8642-d5bcaf9aa2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:34.823396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f7a90-d52f-4733-a8e9-143c5e7da173 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:34.823875 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b879f72-f1b0-4add-8642-d5bcaf9aa2e4 None admin] PATH_INFO: `/v3/services/network` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:34.824668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b879f72-f1b0-4add-8642-d5bcaf9aa2e4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:34.826865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b879f72-f1b0-4add-8642-d5bcaf9aa2e4 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-9b879f72-f1b0-4add-8642-d5bcaf9aa2e4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:34.846609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:34.847707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:34.847707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:34.847707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:34.847707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:34.847707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:34.847707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:34.847707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:34.847707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:34.847707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:34.847707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:34.847707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:34.847707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:34.847707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:34.847707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:34.847707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:34.847707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:34.848584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:34.848584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:34.848584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:34.848584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:34.848584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:34.848584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:34.848584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:34.848584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:34.848584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:34.848584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:34.848584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:34.848584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:34.848584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 07:54:34.848584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:34.852272 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 379/779] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:34 2022] GET /identity/v3/services/network => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:34.853320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b879f72-f1b0-4add-8642-d5bcaf9aa2e4 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:34.944944 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:34.957519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:34.967354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f7a90-d52f-4733-a8e9-143c5e7da173 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:34.986301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b879f72-f1b0-4add-8642-d5bcaf9aa2e4 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:35.000023 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e25802-f90a-480f-9740-626eac599991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:35.001028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e25802-f90a-480f-9740-626eac599991 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:35.001516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e25802-f90a-480f-9740-626eac599991 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:35.002096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e25802-f90a-480f-9740-626eac599991 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:35.003083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e25802-f90a-480f-9740-626eac599991 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:35.003615 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e25802-f90a-480f-9740-626eac599991 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:35.005461 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e25802-f90a-480f-9740-626eac599991 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:35.020810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 401/780] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:35.022701 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e25802-f90a-480f-9740-626eac599991 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:35.037259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd29a10-14c5-45be-baaa-b392600102b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:35.038121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd29a10-14c5-45be-baaa-b392600102b0 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:35.038404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd29a10-14c5-45be-baaa-b392600102b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:35.038670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd29a10-14c5-45be-baaa-b392600102b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:35.047493 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd29a10-14c5-45be-baaa-b392600102b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:35.047493 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd29a10-14c5-45be-baaa-b392600102b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:35.047493 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd29a10-14c5-45be-baaa-b392600102b0 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:35.059101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 380/781] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:35.060216 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd29a10-14c5-45be-baaa-b392600102b0 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:35.170745 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e25802-f90a-480f-9740-626eac599991 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:35.174751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd29a10-14c5-45be-baaa-b392600102b0 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:35.212842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2496a6d-31bc-44fa-809c-c4d4e49568b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:35.213448 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2496a6d-31bc-44fa-809c-c4d4e49568b2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:35.213724 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2496a6d-31bc-44fa-809c-c4d4e49568b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:35.213992 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2496a6d-31bc-44fa-809c-c4d4e49568b2 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:35.214622 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2496a6d-31bc-44fa-809c-c4d4e49568b2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:35.214922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2496a6d-31bc-44fa-809c-c4d4e49568b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:35.216346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2496a6d-31bc-44fa-809c-c4d4e49568b2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:35.221302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 402/782] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Oct 6 07:54:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:35.222196 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2496a6d-31bc-44fa-809c-c4d4e49568b2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:35.234964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f8587-cb8f-4fc5-9153-a793059c964d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:35.234964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f8587-cb8f-4fc5-9153-a793059c964d None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:35.234964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f8587-cb8f-4fc5-9153-a793059c964d None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:35.234964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f8587-cb8f-4fc5-9153-a793059c964d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:35.235383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f8587-cb8f-4fc5-9153-a793059c964d None admin] RBAC: Adding query filter params (role.id=9f0e36c2ed8843fbabd3a4cf30bc3725, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=0bd9c73ff2d94089a34de12c1155e1b5) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:35.235383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f8587-cb8f-4fc5-9153-a793059c964d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:35.235383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f8587-cb8f-4fc5-9153-a793059c964d None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:35.238792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:35.240488 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:35.243010 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 381/783] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:35 2022] GET /identity/v3/role_assignments?user.id=0bd9c73ff2d94089a34de12c1155e1b5&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:35.251399 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f8587-cb8f-4fc5-9153-a793059c964d None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:35.420956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2496a6d-31bc-44fa-809c-c4d4e49568b2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:35.453790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f8587-cb8f-4fc5-9153-a793059c964d None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:35.462084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04a1f8-cb11-4517-a9d5-ebf158ad278d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:35.462802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04a1f8-cb11-4517-a9d5-ebf158ad278d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:35.463115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04a1f8-cb11-4517-a9d5-ebf158ad278d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:35.463384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04a1f8-cb11-4517-a9d5-ebf158ad278d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:35.471886 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04a1f8-cb11-4517-a9d5-ebf158ad278d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:35.473326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04a1f8-cb11-4517-a9d5-ebf158ad278d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:35.485204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-7b04a1f8-cb11-4517-a9d5-ebf158ad278d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 07:54:35.485204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:35.485204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:35.485204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:35.485204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:35.485204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:35.485204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:35.485204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:35.485204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:35.485204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:35.485204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:35.485204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:35.485204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:35.485204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:35.485204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:35.485204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:35.486167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:35.487141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:35.487141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:35.487141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:35.487141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:35.487141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:35.487141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:35.487141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:35.487141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:35.487141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 07:54:35.487141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:35.487141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 403/784] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:54:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:35.492120 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52243a34-4cfd-4d8e-a184-b1fa5833d3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:35.492523 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717d46d-99cd-4b39-9688-139a15876b8a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:35.492806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717d46d-99cd-4b39-9688-139a15876b8a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:35.493018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52243a34-4cfd-4d8e-a184-b1fa5833d3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:35.493125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717d46d-99cd-4b39-9688-139a15876b8a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:35.493451 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52243a34-4cfd-4d8e-a184-b1fa5833d3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:35.493810 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52243a34-4cfd-4d8e-a184-b1fa5833d3c7 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:35.494537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52243a34-4cfd-4d8e-a184-b1fa5833d3c7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:35.494640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 404/785] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:35.495326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717d46d-99cd-4b39-9688-139a15876b8a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:35.499159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52243a34-4cfd-4d8e-a184-b1fa5833d3c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:35.500623 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52243a34-4cfd-4d8e-a184-b1fa5833d3c7 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:35.511043 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 382/786] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Oct 6 07:54:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:35.512419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a38d6-2997-4d31-bf79-3296f14fdfae None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:35.512708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a38d6-2997-4d31-bf79-3296f14fdfae None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:35.512981 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a38d6-2997-4d31-bf79-3296f14fdfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:35.531406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-979a38d6-2997-4d31-bf79-3296f14fdfae None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:35.532587 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-979a38d6-2997-4d31-bf79-3296f14fdfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:35.588010 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:35.608216 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:35.623047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717d46d-99cd-4b39-9688-139a15876b8a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:35.623519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 383/787] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:54:35.624536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a38d6-2997-4d31-bf79-3296f14fdfae None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:35.687276 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f4b54-7012-4778-881b-b94bf17f2511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:35.688114 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f4b54-7012-4778-881b-b94bf17f2511 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:35.690495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f4b54-7012-4778-881b-b94bf17f2511 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:35.690956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f4b54-7012-4778-881b-b94bf17f2511 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:35.691863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f4b54-7012-4778-881b-b94bf17f2511 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:35.692326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f4b54-7012-4778-881b-b94bf17f2511 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:35.697556 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f4b54-7012-4778-881b-b94bf17f2511 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:35.726038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 405/788] 10.222.0.30 () {56 vars in 1135 bytes} [Thu Oct 6 07:54:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:35.733810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f4b54-7012-4778-881b-b94bf17f2511 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:35.815388 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a38d6-2997-4d31-bf79-3296f14fdfae None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:35.883360 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f31bc-fa3b-41c7-ac5d-d643240a79ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:35.884227 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f31bc-fa3b-41c7-ac5d-d643240a79ce None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:35.884656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f31bc-fa3b-41c7-ac5d-d643240a79ce None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:35.885077 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f31bc-fa3b-41c7-ac5d-d643240a79ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:35.885958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39f31bc-fa3b-41c7-ac5d-d643240a79ce None admin] RBAC: Adding query filter params (interface=public, service_id=bd5d8b4a455f47a49b28c723c0df72dc, region_id=RegionOne) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:35.886414 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39f31bc-fa3b-41c7-ac5d-d643240a79ce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:35.889087 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39f31bc-fa3b-41c7-ac5d-d643240a79ce None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:35.899360 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 384/789] 10.222.0.30 () {56 vars in 1251 bytes} [Thu Oct 6 07:54:35 2022] GET /identity/v3/endpoints?service_id=bd5d8b4a455f47a49b28c723c0df72dc&interface=public®ion_id=RegionOne => generated 194 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:35.908233 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f31bc-fa3b-41c7-ac5d-d643240a79ce None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:35.911265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f4b54-7012-4778-881b-b94bf17f2511 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:35.966863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:35.979995 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a42765-886f-425d-b267-b081c7d282b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:35.981084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a42765-886f-425d-b267-b081c7d282b8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:35.981084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a42765-886f-425d-b267-b081c7d282b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:35.981356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a42765-886f-425d-b267-b081c7d282b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:35.993227 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a42765-886f-425d-b267-b081c7d282b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:35.994691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a42765-886f-425d-b267-b081c7d282b8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:36.002494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:36.012909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c0a42765-886f-425d-b267-b081c7d282b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:36.012909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:36.012909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:36.012909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:36.012909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:36.012909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:36.012909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:36.012909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:36.012909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:36.012909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:36.012909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:36.012909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:36.012909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:36.012909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:36.012909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:36.012909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:36.013935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:36.019603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:36.019603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:36.019603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:36.019603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 406/790] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:36.019603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a42765-886f-425d-b267-b081c7d282b8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:36.062756 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f31bc-fa3b-41c7-ac5d-d643240a79ce None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:36.130262 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a42765-886f-425d-b267-b081c7d282b8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:36.136719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53305f62-8eca-4bb5-893d-e38d816b0f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:36.137557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53305f62-8eca-4bb5-893d-e38d816b0f1f None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:36.138007 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53305f62-8eca-4bb5-893d-e38d816b0f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:36.138448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53305f62-8eca-4bb5-893d-e38d816b0f1f None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:36.139369 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53305f62-8eca-4bb5-893d-e38d816b0f1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:36.141131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53305f62-8eca-4bb5-893d-e38d816b0f1f None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:36.148027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 385/791] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Oct 6 07:54:35 2022] GET /identity/v3/services => generated 772 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:36.149338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53305f62-8eca-4bb5-893d-e38d816b0f1f None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:36.173355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7232ed-b020-436f-9396-ed8059e29714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:36.173355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7232ed-b020-436f-9396-ed8059e29714 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:36.173355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7232ed-b020-436f-9396-ed8059e29714 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:36.173355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7232ed-b020-436f-9396-ed8059e29714 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:36.191219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7232ed-b020-436f-9396-ed8059e29714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:36.191219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1b7232ed-b020-436f-9396-ed8059e29714 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:54:36.191219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:36.191219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:36.191219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:36.191219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:36.191219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:36.191219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:36.191219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:36.191219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:36.191219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:36.191219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:36.191219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:36.191219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:36.192299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:36.194097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:36.194097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:36.194097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:36.194097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:36.194097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:36.194097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:36.194097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:36.194097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:36.194097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:36.194097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:36.194097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:36.194097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:36.194097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:54:36.194097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:36.194097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 407/792] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:54:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:36.209590 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7232ed-b020-436f-9396-ed8059e29714 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:36.307996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:36.380779 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53305f62-8eca-4bb5-893d-e38d816b0f1f None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:36.392843 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7232ed-b020-436f-9396-ed8059e29714 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:36.417660 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:36.432489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe98def-1c41-4de1-a583-9483ed899917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:36.433094 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe98def-1c41-4de1-a583-9483ed899917 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:36.433365 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe98def-1c41-4de1-a583-9483ed899917 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:36.433629 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe98def-1c41-4de1-a583-9483ed899917 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:36.434270 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe98def-1c41-4de1-a583-9483ed899917 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:36.434563 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe98def-1c41-4de1-a583-9483ed899917 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:36.437315 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe98def-1c41-4de1-a583-9483ed899917 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:36.441983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014de78-75c5-454f-bc0a-e1931d50a55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:36.442584 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014de78-75c5-454f-bc0a-e1931d50a55f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:36.442898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014de78-75c5-454f-bc0a-e1931d50a55f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:36.443196 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014de78-75c5-454f-bc0a-e1931d50a55f None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:36.443982 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014de78-75c5-454f-bc0a-e1931d50a55f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:36.444232 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014de78-75c5-454f-bc0a-e1931d50a55f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:36.445579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014de78-75c5-454f-bc0a-e1931d50a55f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:36.458111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 408/793] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:54:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:36.459728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc90a4-969f-44d2-8687-6be25a0b427d None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:36.460229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc90a4-969f-44d2-8687-6be25a0b427d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:36.460707 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc90a4-969f-44d2-8687-6be25a0b427d None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:36.462405 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 409/794] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:36.466530 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 386/795] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:36.467916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0471f8-ac87-4be3-87fd-8b461d9260b2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:36.468436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0471f8-ac87-4be3-87fd-8b461d9260b2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:36.468916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0471f8-ac87-4be3-87fd-8b461d9260b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:36.469115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe98def-1c41-4de1-a583-9483ed899917 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:36.496989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-be0471f8-ac87-4be3-87fd-8b461d9260b2 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:36.498168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-be0471f8-ac87-4be3-87fd-8b461d9260b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:36.582260 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 410/796] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:54:36.582260 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0471f8-ac87-4be3-87fd-8b461d9260b2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:36.614063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe98def-1c41-4de1-a583-9483ed899917 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:36.641354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:36.679003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35910d-9d7c-473a-bf5b-b397ee91f3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:36.679626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35910d-9d7c-473a-bf5b-b397ee91f3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:36.679898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35910d-9d7c-473a-bf5b-b397ee91f3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:36.680163 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35910d-9d7c-473a-bf5b-b397ee91f3b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:36.690784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35910d-9d7c-473a-bf5b-b397ee91f3b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:36.692235 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35910d-9d7c-473a-bf5b-b397ee91f3b9 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:36.704179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-fc35910d-9d7c-473a-bf5b-b397ee91f3b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:36.704179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:36.704179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:36.704179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:36.704179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:36.704179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:36.704179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:36.704179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:36.704179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:36.704179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:36.704179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:36.704179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:36.704179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:36.704179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:36.704179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:36.705109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:36.706061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:36.706061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:36.706061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:36.706061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:36.706061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:36.706061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:36.706061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:36.706061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:36.706061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 387/797] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:36.713779 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35910d-9d7c-473a-bf5b-b397ee91f3b9 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:36.745561 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0471f8-ac87-4be3-87fd-8b461d9260b2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:36.783064 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2af58-4c8c-4c2a-bdc3-150974eddf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:36.784807 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2af58-4c8c-4c2a-bdc3-150974eddf06 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:36.785181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2af58-4c8c-4c2a-bdc3-150974eddf06 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:36.785543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2af58-4c8c-4c2a-bdc3-150974eddf06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:36.796792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2af58-4c8c-4c2a-bdc3-150974eddf06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:36.798352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2af58-4c8c-4c2a-bdc3-150974eddf06 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:36.800910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:36.815589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5cd2af58-4c8c-4c2a-bdc3-150974eddf06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:36.815589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:36.815589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:36.815589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:36.815589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:36.815589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:36.815589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:36.815589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:36.815589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:36.815589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:36.815589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:36.815589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:36.815589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:36.815589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:36.815589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:36.815589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:36.819201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:36.820145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:36.820145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:36.820145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:36.820145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 411/798] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:36.823722 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2af58-4c8c-4c2a-bdc3-150974eddf06 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:36.919500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:36.920793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35910d-9d7c-473a-bf5b-b397ee91f3b9 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:36.940360 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2af58-4c8c-4c2a-bdc3-150974eddf06 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:36.961971 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d58a15-8f3e-4a93-a41c-3755ac2be2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:36.962576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d58a15-8f3e-4a93-a41c-3755ac2be2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:36.962849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d58a15-8f3e-4a93-a41c-3755ac2be2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:36.963317 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d58a15-8f3e-4a93-a41c-3755ac2be2b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:36.964537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d58a15-8f3e-4a93-a41c-3755ac2be2b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:36.964846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d58a15-8f3e-4a93-a41c-3755ac2be2b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:36.966569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d58a15-8f3e-4a93-a41c-3755ac2be2b2 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:36.979136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0169e06-d691-42f2-9ae6-4ec6a0e17007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:36.979736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0169e06-d691-42f2-9ae6-4ec6a0e17007 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:36.980008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0169e06-d691-42f2-9ae6-4ec6a0e17007 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:36.980273 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0169e06-d691-42f2-9ae6-4ec6a0e17007 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:36.980913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0169e06-d691-42f2-9ae6-4ec6a0e17007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:36.981210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0169e06-d691-42f2-9ae6-4ec6a0e17007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:36.982556 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0169e06-d691-42f2-9ae6-4ec6a0e17007 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:36.987082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 388/799] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:36.987941 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d58a15-8f3e-4a93-a41c-3755ac2be2b2 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:37.001369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 412/800] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:37.002233 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0169e06-d691-42f2-9ae6-4ec6a0e17007 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:37.121239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0169e06-d691-42f2-9ae6-4ec6a0e17007 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:37.121436 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d58a15-8f3e-4a93-a41c-3755ac2be2b2 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:37.123091 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:37.164175 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9a046-48a5-4c7d-be9c-54beb104de34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:37.165152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9a046-48a5-4c7d-be9c-54beb104de34 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:37.165657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9a046-48a5-4c7d-be9c-54beb104de34 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:37.166240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9a046-48a5-4c7d-be9c-54beb104de34 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:37.167145 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9a046-48a5-4c7d-be9c-54beb104de34 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:37.168849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9a046-48a5-4c7d-be9c-54beb104de34 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:37.169644 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe4986-af75-4c7b-a95a-2e33fdedd9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:37.170375 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe4986-af75-4c7b-a95a-2e33fdedd9b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:37.170746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe4986-af75-4c7b-a95a-2e33fdedd9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:37.171142 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe4986-af75-4c7b-a95a-2e33fdedd9b5 None admin] PATH_INFO: `/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/3fa1c430c9f7416eaf1d17300b827e05/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:37.174062 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe4986-af75-4c7b-a95a-2e33fdedd9b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=399f0095cc8b4c64893ccc75af13c52d, user_id=3fa1c430c9f7416eaf1d17300b827e05, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:37.175602 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe4986-af75-4c7b-a95a-2e33fdedd9b5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-9ad9a046-48a5-4c7d-be9c-54beb104de34 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:37.180335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:37.181375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:37.181375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:37.181375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:37.181375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:37.181375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:37.181375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:37.181375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:37.181375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:37.181375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:37.181375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:37.181375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:37.181375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:37.181375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:37.181375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:37.181375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:37.181375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:37.182310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:37.182310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:37.182310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:37.182310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:37.182310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:37.182310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:37.182310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:37.182310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:37.182310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:37.182310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:37.182310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:37.182310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:37.182310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 06 07:54:37.182310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:37.186792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 389/801] 10.222.0.30 () {56 vars in 1129 bytes} [Thu Oct 6 07:54:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:37.187715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9a046-48a5-4c7d-be9c-54beb104de34 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:37.235503 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 413/802] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:54:37 2022] PUT /identity/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/3fa1c430c9f7416eaf1d17300b827e05/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:54:37.236684 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe4986-af75-4c7b-a95a-2e33fdedd9b5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:37.246343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:37.389321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9a046-48a5-4c7d-be9c-54beb104de34 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:37.424665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe4986-af75-4c7b-a95a-2e33fdedd9b5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:37.461622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce28b0-dfda-4107-a651-9d2e99abce25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:37.462775 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce28b0-dfda-4107-a651-9d2e99abce25 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:37.463492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce28b0-dfda-4107-a651-9d2e99abce25 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:37.463927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce28b0-dfda-4107-a651-9d2e99abce25 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:37.477080 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce28b0-dfda-4107-a651-9d2e99abce25 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:37.477405 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cd833-d849-4914-902a-067ef79e29ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:37.478019 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cd833-d849-4914-902a-067ef79e29ad None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:37.478292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cd833-d849-4914-902a-067ef79e29ad None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:37.478563 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cd833-d849-4914-902a-067ef79e29ad None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:37.479726 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539cd833-d849-4914-902a-067ef79e29ad None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:37.479726 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539cd833-d849-4914-902a-067ef79e29ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:37.480497 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce28b0-dfda-4107-a651-9d2e99abce25 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:37.480881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539cd833-d849-4914-902a-067ef79e29ad None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:37.486984 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 414/803] 10.222.0.30 () {56 vars in 1124 bytes} [Thu Oct 6 07:54:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:37.492819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cd833-d849-4914-902a-067ef79e29ad None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:37.499851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-4bce28b0-dfda-4107-a651-9d2e99abce25 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:54:37.499851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:37.499851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:37.499851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:37.499851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:37.499851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:37.499851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:37.499851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:37.499851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:37.499851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:37.499851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:37.499851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:37.499851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:37.499851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:37.499851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:37.499851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:37.500813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:37.501780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:37.501780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:37.501780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:37.501780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:37.501780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:37.501780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:37.501780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:37.501780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:37.501780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:54:37.501780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:37.501780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 390/804] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Oct 6 07:54:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:37.507791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce28b0-dfda-4107-a651-9d2e99abce25 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:37.555129 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:37.586623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:37.651574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cd833-d849-4914-902a-067ef79e29ad None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:37.683923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce28b0-dfda-4107-a651-9d2e99abce25 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:37.711816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a6853-c90c-4e04-84ce-366b48348903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:37.712426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a6853-c90c-4e04-84ce-366b48348903 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:37.712697 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a6853-c90c-4e04-84ce-366b48348903 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:37.712962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a6853-c90c-4e04-84ce-366b48348903 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:37.713592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a6853-c90c-4e04-84ce-366b48348903 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:37.713886 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a6853-c90c-4e04-84ce-366b48348903 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:37.715944 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a6853-c90c-4e04-84ce-366b48348903 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:37.721966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 415/805] 10.222.0.30 () {56 vars in 1124 bytes} [Thu Oct 6 07:54:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:37.723336 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3373dd-cf33-439f-8b07-0ed7278ca631 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:37.723622 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3373dd-cf33-439f-8b07-0ed7278ca631 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:37.723941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3373dd-cf33-439f-8b07-0ed7278ca631 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:37.729812 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 416/806] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:37.730889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3373dd-cf33-439f-8b07-0ed7278ca631 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:37.741310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5f93c-d7e0-408e-91bc-35341e032a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:37.741919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5f93c-d7e0-408e-91bc-35341e032a55 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:37.742191 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5f93c-d7e0-408e-91bc-35341e032a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:37.742462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5f93c-d7e0-408e-91bc-35341e032a55 None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:37.743689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5f93c-d7e0-408e-91bc-35341e032a55 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:37.743989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5f93c-d7e0-408e-91bc-35341e032a55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:37.745777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5f93c-d7e0-408e-91bc-35341e032a55 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:37.769252 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 391/807] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Oct 6 07:54:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:37.770627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5366beb-89e6-4444-8377-8d42497e5637 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:37.771619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5366beb-89e6-4444-8377-8d42497e5637 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:37.771897 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5366beb-89e6-4444-8377-8d42497e5637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:37.793583 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-e5366beb-89e6-4444-8377-8d42497e5637 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:37.795300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e5366beb-89e6-4444-8377-8d42497e5637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:37.863801 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3373dd-cf33-439f-8b07-0ed7278ca631 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:37.879941 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 392/808] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:54:37.881429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5366beb-89e6-4444-8377-8d42497e5637 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:37.898669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:37.919373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:37.935749 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715366e-9d63-48d4-bab6-c09456dca59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:37.936377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715366e-9d63-48d4-bab6-c09456dca59f None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:37.936630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715366e-9d63-48d4-bab6-c09456dca59f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:37.936828 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715366e-9d63-48d4-bab6-c09456dca59f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:37.937479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715366e-9d63-48d4-bab6-c09456dca59f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:37.938916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715366e-9d63-48d4-bab6-c09456dca59f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:37.992554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5366beb-89e6-4444-8377-8d42497e5637 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:38.029657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee0ad1-0d51-4bb9-a81b-8279bec6bf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:38.030347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee0ad1-0d51-4bb9-a81b-8279bec6bf69 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:38.030625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee0ad1-0d51-4bb9-a81b-8279bec6bf69 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:38.030951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee0ad1-0d51-4bb9-a81b-8279bec6bf69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:38.062234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 417/809] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 07:54:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:54:38.108390 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee0ad1-0d51-4bb9-a81b-8279bec6bf69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:38.109848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee0ad1-0d51-4bb9-a81b-8279bec6bf69 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:38.127079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:38.228122 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 393/810] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 07:54:38.233775 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715366e-9d63-48d4-bab6-c09456dca59f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:38.318907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee0ad1-0d51-4bb9-a81b-8279bec6bf69 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:38.378669 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715366e-9d63-48d4-bab6-c09456dca59f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:38.426338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f9e4b-28ea-47be-b499-6aeba7342cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:38.428813 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f9e4b-28ea-47be-b499-6aeba7342cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:38.428813 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f9e4b-28ea-47be-b499-6aeba7342cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:38.428813 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f9e4b-28ea-47be-b499-6aeba7342cc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:38.440187 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f9e4b-28ea-47be-b499-6aeba7342cc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:38.441936 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f9e4b-28ea-47be-b499-6aeba7342cc2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:38.458104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-7b3f9e4b-28ea-47be-b499-6aeba7342cc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:38.458104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:38.458104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:38.458104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:38.458104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:38.458104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:38.458104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:38.458104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:38.458104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:38.458104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:38.458104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:38.458104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:38.458104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:38.458104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:38.458104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:38.458104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:38.459464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:38.460372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:38.460372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:38.460372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:38.462414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 418/811] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:38.466137 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee0ad1-0d51-4bb9-a81b-8279bec6bf69 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:38.468770 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f9e4b-28ea-47be-b499-6aeba7342cc2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:38.483637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:38.508261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dfc8b-a25b-4077-b45b-af03b0ae9024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:38.508914 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dfc8b-a25b-4077-b45b-af03b0ae9024 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:38.509193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dfc8b-a25b-4077-b45b-af03b0ae9024 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:38.509461 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dfc8b-a25b-4077-b45b-af03b0ae9024 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:38.512884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:38.523418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8dfc8b-a25b-4077-b45b-af03b0ae9024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:38.528566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-de8dfc8b-a25b-4077-b45b-af03b0ae9024 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:38.528566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:38.528566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:38.528566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:38.528566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:38.528566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:38.528566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:38.528566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:38.528566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:38.528566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:38.528566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:38.528566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:38.528566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:38.528566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:38.528566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:38.528566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:38.530260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:38.530260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:38.530260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:38.530260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:38.530260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:38.530260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:38.530260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:38.530260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:38.530260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:38.530260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:38.530260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:38.530260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:38.530260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:38.530260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:38.530260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:38.530260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:38.531297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:38.531297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:38.531297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:38.531297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:38.531297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:38.531297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:38.531297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:38.531297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:38.531297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:38.531297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:38.531297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:38.531297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:38.533297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 394/812] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:38.543950 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dfc8b-a25b-4077-b45b-af03b0ae9024 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:38.595487 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f9e4b-28ea-47be-b499-6aeba7342cc2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:38.642766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ff02a-5795-411e-bb12-8837b0c61aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:38.643645 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ff02a-5795-411e-bb12-8837b0c61aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:38.644491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ff02a-5795-411e-bb12-8837b0c61aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:38.644911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ff02a-5795-411e-bb12-8837b0c61aa5 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:38.647158 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ff02a-5795-411e-bb12-8837b0c61aa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:38.647760 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ff02a-5795-411e-bb12-8837b0c61aa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:38.649381 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ff02a-5795-411e-bb12-8837b0c61aa5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:38.664375 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 419/813] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:38.666114 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b7a90-6ec7-49f7-aeb0-706ee15478aa None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:38.666544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b7a90-6ec7-49f7-aeb0-706ee15478aa None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:38.666984 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b7a90-6ec7-49f7-aeb0-706ee15478aa None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:38.668667 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 420/814] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:38.669787 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b7a90-6ec7-49f7-aeb0-706ee15478aa None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:38.674739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dfc8b-a25b-4077-b45b-af03b0ae9024 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:38.725782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cef989-b961-4401-9779-2d286e3af31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:38.727121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cef989-b961-4401-9779-2d286e3af31c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:38.727601 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cef989-b961-4401-9779-2d286e3af31c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:38.728394 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cef989-b961-4401-9779-2d286e3af31c None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:38.729547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cef989-b961-4401-9779-2d286e3af31c None admin] RBAC: Adding query filter params (name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:38.730141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cef989-b961-4401-9779-2d286e3af31c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:38.732411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cef989-b961-4401-9779-2d286e3af31c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:38.749249 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 395/815] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:38.749249 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d707d-d0c1-4046-9b71-48509d2e4ffb None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:38.749249 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d707d-d0c1-4046-9b71-48509d2e4ffb None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:38.749249 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d707d-d0c1-4046-9b71-48509d2e4ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:38.768287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-cf6d707d-d0c1-4046-9b71-48509d2e4ffb None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:38.769593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-cf6d707d-d0c1-4046-9b71-48509d2e4ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:38.790697 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:38.798858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b7a90-6ec7-49f7-aeb0-706ee15478aa None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:38.801602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:38.843341 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42edfed-0738-4416-8033-0a4b8f42b39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:38.843973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42edfed-0738-4416-8033-0a4b8f42b39f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:38.844244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42edfed-0738-4416-8033-0a4b8f42b39f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:38.844511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42edfed-0738-4416-8033-0a4b8f42b39f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:38.851210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 396/816] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:54:38.851580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42edfed-0738-4416-8033-0a4b8f42b39f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:38.852603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d707d-d0c1-4046-9b71-48509d2e4ffb None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:38.853340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42edfed-0738-4416-8033-0a4b8f42b39f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:38.867798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a42edfed-0738-4416-8033-0a4b8f42b39f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:38.867798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:38.867798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:38.867798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:38.867798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:38.867798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:38.867798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:38.867798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:38.867798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:38.867798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:38.867798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:38.867798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:38.867798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:38.867798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:38.867798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:38.868948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:38.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:38.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:38.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:38.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:38.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:38.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:38.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:38.870074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:38.875015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 421/817] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:38.876576 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42edfed-0738-4416-8033-0a4b8f42b39f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:39.010038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d707d-d0c1-4046-9b71-48509d2e4ffb None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:39.050434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9e439-a115-4042-b0e0-62ee69368ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:39.051088 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9e439-a115-4042-b0e0-62ee69368ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:39.055047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9e439-a115-4042-b0e0-62ee69368ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:39.055047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9e439-a115-4042-b0e0-62ee69368ec2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:39.063609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9e439-a115-4042-b0e0-62ee69368ec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:39.065063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9e439-a115-4042-b0e0-62ee69368ec2 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:39.082413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-88e9e439-a115-4042-b0e0-62ee69368ec2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:39.082413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:39.082413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:39.082413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:39.082413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:39.082413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:39.082413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:39.082413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:39.082413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:39.082413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:39.082413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:39.082413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:39.082413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:39.082413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:39.082413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:39.082413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:39.084793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:39.085698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:39.085698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:39.085698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:39.089402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 397/818] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:39.090747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9e439-a115-4042-b0e0-62ee69368ec2 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:39.112675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:39.115321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42edfed-0738-4416-8033-0a4b8f42b39f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:39.157816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6673d0-655a-4c42-95a4-670564cdc22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:39.158432 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6673d0-655a-4c42-95a4-670564cdc22f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:39.158707 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6673d0-655a-4c42-95a4-670564cdc22f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:39.159004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6673d0-655a-4c42-95a4-670564cdc22f None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:39.159704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6673d0-655a-4c42-95a4-670564cdc22f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:39.160013 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6673d0-655a-4c42-95a4-670564cdc22f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:39.161958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6673d0-655a-4c42-95a4-670564cdc22f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:39.174347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 422/819] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:39.175271 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6673d0-655a-4c42-95a4-670564cdc22f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:39.195501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:39.225137 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9e439-a115-4042-b0e0-62ee69368ec2 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:39.274331 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062303e1-2aff-443c-b454-7baaafa73847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:39.275954 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062303e1-2aff-443c-b454-7baaafa73847 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:39.276441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062303e1-2aff-443c-b454-7baaafa73847 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:39.277151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062303e1-2aff-443c-b454-7baaafa73847 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:39.278414 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062303e1-2aff-443c-b454-7baaafa73847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:39.279239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062303e1-2aff-443c-b454-7baaafa73847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:39.282661 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062303e1-2aff-443c-b454-7baaafa73847 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:39.283693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6673d0-655a-4c42-95a4-670564cdc22f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:39.302684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 398/820] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:39.302684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062303e1-2aff-443c-b454-7baaafa73847 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:39.342574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d1437-96cf-4603-82fa-202e72d4b452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:39.343840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d1437-96cf-4603-82fa-202e72d4b452 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:39.344314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d1437-96cf-4603-82fa-202e72d4b452 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:39.344888 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d1437-96cf-4603-82fa-202e72d4b452 None admin] PATH_INFO: `/v3/services/network` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:39.345719 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d1437-96cf-4603-82fa-202e72d4b452 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:39.347343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d1437-96cf-4603-82fa-202e72d4b452 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3b3d1437-96cf-4603-82fa-202e72d4b452 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:39.357593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:39.358634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:39.358634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:39.358634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:39.358634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:39.358634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:39.358634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:39.358634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:39.358634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:39.358634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:39.358634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:39.358634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:39.358634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:39.358634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:39.358634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:39.358634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:39.358634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:39.359525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:39.359525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:39.359525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:39.359525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:39.359525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:39.359525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:39.359525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:39.359525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:39.359525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:39.359525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:39.359525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:39.359525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:39.359525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 06 07:54:39.359525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:39.361036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 423/821] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:39 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:39.361379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d1437-96cf-4603-82fa-202e72d4b452 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:39.404967 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:39.433246 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062303e1-2aff-443c-b454-7baaafa73847 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:39.475672 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:39.480669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e078ed-4bba-4910-aabc-ef1df1752ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:39.481270 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e078ed-4bba-4910-aabc-ef1df1752ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:39.481551 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e078ed-4bba-4910-aabc-ef1df1752ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:39.481820 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e078ed-4bba-4910-aabc-ef1df1752ed5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:39.488175 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e078ed-4bba-4910-aabc-ef1df1752ed5 None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=209795285384486183cda455fe316bc9) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:39.488175 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e078ed-4bba-4910-aabc-ef1df1752ed5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:39.488175 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e078ed-4bba-4910-aabc-ef1df1752ed5 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:39.493502 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 399/822] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:39 2022] GET /identity/v3/role_assignments?user.id=209795285384486183cda455fe316bc9&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:39.494434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e078ed-4bba-4910-aabc-ef1df1752ed5 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:39.500656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d1437-96cf-4603-82fa-202e72d4b452 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:39.574332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2f077-36e3-416e-bf31-3bc4abf90bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:39.576417 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2f077-36e3-416e-bf31-3bc4abf90bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:39.576895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2f077-36e3-416e-bf31-3bc4abf90bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:39.577480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2f077-36e3-416e-bf31-3bc4abf90bb9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:39.595052 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2f077-36e3-416e-bf31-3bc4abf90bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:39.597005 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2f077-36e3-416e-bf31-3bc4abf90bb9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:39.622789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-caf2f077-36e3-416e-bf31-3bc4abf90bb9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 07:54:39.622789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:39.622789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:39.622789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:39.622789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:39.622789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:39.622789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:39.622789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:39.622789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:39.622789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:39.622789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:39.622789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:39.622789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:39.622789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:39.622789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:39.622789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:39.624117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:39.625069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:39.625069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:39.625069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:39.625069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:39.625069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:39.625069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:39.625069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:39.625069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:39.625069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 07:54:39.625069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:39.627683 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 424/823] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Oct 6 07:54:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:39.635173 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2f077-36e3-416e-bf31-3bc4abf90bb9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:39.689703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e078ed-4bba-4910-aabc-ef1df1752ed5 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:39.755567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b187f36-d7e7-4a6c-8165-f239c58ddc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:39.756531 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b187f36-d7e7-4a6c-8165-f239c58ddc5d None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:39.757024 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b187f36-d7e7-4a6c-8165-f239c58ddc5d None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:39.757489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b187f36-d7e7-4a6c-8165-f239c58ddc5d None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:39.759006 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b187f36-d7e7-4a6c-8165-f239c58ddc5d None admin] RBAC: Adding query filter params (name=network) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:39.759462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b187f36-d7e7-4a6c-8165-f239c58ddc5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:39.760833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b187f36-d7e7-4a6c-8165-f239c58ddc5d None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:39.766074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 400/824] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Oct 6 07:54:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:39.775927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b187f36-d7e7-4a6c-8165-f239c58ddc5d None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:39.785399 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:39.827581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2f077-36e3-416e-bf31-3bc4abf90bb9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:39.868770 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6aff8c-02f7-427c-9fc2-ddfe9065f55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:39.869411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6aff8c-02f7-427c-9fc2-ddfe9065f55e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:39.869682 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6aff8c-02f7-427c-9fc2-ddfe9065f55e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:39.869948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6aff8c-02f7-427c-9fc2-ddfe9065f55e None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:39.870675 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6aff8c-02f7-427c-9fc2-ddfe9065f55e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:39.870995 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6aff8c-02f7-427c-9fc2-ddfe9065f55e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:39.872979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6aff8c-02f7-427c-9fc2-ddfe9065f55e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:39.887004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:39.901777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 425/825] 10.222.0.30 () {56 vars in 1149 bytes} [Thu Oct 6 07:54:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:39.903206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5fb417-4a88-43e5-a5a3-851bfa680444 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:39.903483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5fb417-4a88-43e5-a5a3-851bfa680444 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:39.903753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5fb417-4a88-43e5-a5a3-851bfa680444 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:39.907253 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b187f36-d7e7-4a6c-8165-f239c58ddc5d None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:39.910803 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 426/826] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:39.912914 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5fb417-4a88-43e5-a5a3-851bfa680444 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:39.961641 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bb21f-e2fb-4aac-a8cc-903851b85ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:39.962841 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bb21f-e2fb-4aac-a8cc-903851b85ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:39.963380 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bb21f-e2fb-4aac-a8cc-903851b85ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:39.963800 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bb21f-e2fb-4aac-a8cc-903851b85ad5 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:39.964605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bb21f-e2fb-4aac-a8cc-903851b85ad5 None admin] RBAC: Adding query filter params (type=network) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:39.965645 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bb21f-e2fb-4aac-a8cc-903851b85ad5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:39.967772 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bb21f-e2fb-4aac-a8cc-903851b85ad5 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:39.979815 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 401/827] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Oct 6 07:54:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:39.980742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab808e-28b9-454a-95ad-c4da4c60abe1 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:39.981029 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab808e-28b9-454a-95ad-c4da4c60abe1 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:39.981302 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab808e-28b9-454a-95ad-c4da4c60abe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:40.000594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-60ab808e-28b9-454a-95ad-c4da4c60abe1 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:40.001781 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-60ab808e-28b9-454a-95ad-c4da4c60abe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:40.046600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5fb417-4a88-43e5-a5a3-851bfa680444 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:40.068531 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:40.074326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 402/828] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:54:40.080007 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab808e-28b9-454a-95ad-c4da4c60abe1 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:40.089364 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b064ce-707f-4272-9e7e-6da922c9d1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:40.090143 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b064ce-707f-4272-9e7e-6da922c9d1ed None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:40.090507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b064ce-707f-4272-9e7e-6da922c9d1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:40.090851 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b064ce-707f-4272-9e7e-6da922c9d1ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:40.102909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b064ce-707f-4272-9e7e-6da922c9d1ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:40.105439 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b064ce-707f-4272-9e7e-6da922c9d1ed None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:40.124314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-11b064ce-707f-4272-9e7e-6da922c9d1ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:40.124314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:40.124314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:40.124314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:40.124314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:40.124314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:40.124314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:40.124314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:40.124314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:40.124314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:40.124314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:40.124314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:40.124314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:40.124314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:40.124314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:40.124314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:40.125696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:40.126603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:40.126603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:40.126603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:40.126603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 427/829] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:40.131116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b064ce-707f-4272-9e7e-6da922c9d1ed None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:40.189465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:40.255586 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b064ce-707f-4272-9e7e-6da922c9d1ed None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:40.284880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab808e-28b9-454a-95ad-c4da4c60abe1 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:40.294020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858264a4-09f5-4ba9-9559-ee17ee5f9eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:40.294632 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858264a4-09f5-4ba9-9559-ee17ee5f9eef None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:40.294910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858264a4-09f5-4ba9-9559-ee17ee5f9eef None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:40.295210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858264a4-09f5-4ba9-9559-ee17ee5f9eef None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:40.295856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858264a4-09f5-4ba9-9559-ee17ee5f9eef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:40.296156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858264a4-09f5-4ba9-9559-ee17ee5f9eef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:40.297581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858264a4-09f5-4ba9-9559-ee17ee5f9eef None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:40.314055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 428/830] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:40.331192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858264a4-09f5-4ba9-9559-ee17ee5f9eef None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:40.342219 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436ccac-e51c-4bee-b733-22d4dd07d83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:40.342827 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436ccac-e51c-4bee-b733-22d4dd07d83c None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:40.343135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436ccac-e51c-4bee-b733-22d4dd07d83c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:40.343405 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436ccac-e51c-4bee-b733-22d4dd07d83c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:40.344073 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436ccac-e51c-4bee-b733-22d4dd07d83c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:40.345447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436ccac-e51c-4bee-b733-22d4dd07d83c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:40.439673 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 403/831] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 07:54:40 2022] POST /identity/v3/endpoints => generated 337 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:54:40.447919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436ccac-e51c-4bee-b733-22d4dd07d83c None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:40.463334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858264a4-09f5-4ba9-9559-ee17ee5f9eef None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:40.479723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:40.519970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22abab-6505-48c1-b4bb-bce975270503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:40.522126 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22abab-6505-48c1-b4bb-bce975270503 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:40.523123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22abab-6505-48c1-b4bb-bce975270503 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:40.524624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22abab-6505-48c1-b4bb-bce975270503 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:40.543046 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22abab-6505-48c1-b4bb-bce975270503 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:40.547728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22abab-6505-48c1-b4bb-bce975270503 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:40.579198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-bd22abab-6505-48c1-b4bb-bce975270503 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:40.579198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:40.579198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:40.579198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:40.579198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:40.579198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:40.579198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:40.579198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:40.579198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:40.579198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:40.579198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:40.579198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:40.579198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:40.579198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:40.579198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:40.580186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:40.581113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:40.581113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:40.581113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:40.581113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:40.581113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:40.581113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:40.581113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:40.581113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:40.586783 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 429/832] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:40.589529 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22abab-6505-48c1-b4bb-bce975270503 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:40.606241 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:40.701483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436ccac-e51c-4bee-b733-22d4dd07d83c None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:40.746114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1a003-5066-4774-8fd4-da535050e866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:40.746810 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1a003-5066-4774-8fd4-da535050e866 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:40.747131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1a003-5066-4774-8fd4-da535050e866 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:40.747406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1a003-5066-4774-8fd4-da535050e866 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:40.763817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1a003-5066-4774-8fd4-da535050e866 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:40.765049 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22abab-6505-48c1-b4bb-bce975270503 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:40.770348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-0ae1a003-5066-4774-8fd4-da535050e866 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:54:40.770348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:40.770348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:40.770348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:40.770348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:40.770348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:40.770348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:40.770348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:40.770348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:40.770348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:40.770348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:40.770348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:40.770348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:40.770348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:40.770348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:40.770348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:40.772300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:40.772300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:40.772300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:40.772300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:40.772300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:40.772300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:40.772300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:40.772300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:40.772300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:40.772300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:40.772300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:40.772300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:40.772300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:40.772300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:40.772300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:40.772300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:40.773154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:40.773154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:40.773154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:40.773154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:40.773154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:40.773154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:40.773154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:40.773154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:40.773154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:40.773154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:40.773154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:54:40.773154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:40.773154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 404/833] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:54:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:40.778720 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1a003-5066-4774-8fd4-da535050e866 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:40.833656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d1a3d-f456-4e5e-a4e3-ba30504f15c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:40.834263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d1a3d-f456-4e5e-a4e3-ba30504f15c6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:40.834539 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d1a3d-f456-4e5e-a4e3-ba30504f15c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:40.834810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d1a3d-f456-4e5e-a4e3-ba30504f15c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:40.835716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d1a3d-f456-4e5e-a4e3-ba30504f15c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:40.840591 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d1a3d-f456-4e5e-a4e3-ba30504f15c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:40.840591 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d1a3d-f456-4e5e-a4e3-ba30504f15c6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:40.854320 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 430/834] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:40.855442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653f011-12b0-4b46-a30b-64f91dc9717d None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:40.855721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653f011-12b0-4b46-a30b-64f91dc9717d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:40.855995 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653f011-12b0-4b46-a30b-64f91dc9717d None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:40.857328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 431/835] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:40.862011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653f011-12b0-4b46-a30b-64f91dc9717d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:40.893247 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:40.977699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653f011-12b0-4b46-a30b-64f91dc9717d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:41.015801 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f7cf1-719a-445a-9763-de201d8c273f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:41.016455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1a003-5066-4774-8fd4-da535050e866 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:41.016567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f7cf1-719a-445a-9763-de201d8c273f None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:41.016746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f7cf1-719a-445a-9763-de201d8c273f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:41.016997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f7cf1-719a-445a-9763-de201d8c273f None admin] PATH_INFO: `/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/0bd9c73ff2d94089a34de12c1155e1b5/roles/9f0e36c2ed8843fbabd3a4cf30bc3725` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:41.033248 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f7cf1-719a-445a-9763-de201d8c273f None admin] RBAC: Authorizing `identity:create_grant(project_id=399f0095cc8b4c64893ccc75af13c52d, user_id=0bd9c73ff2d94089a34de12c1155e1b5, role_id=9f0e36c2ed8843fbabd3a4cf30bc3725)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:41.034673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f7cf1-719a-445a-9763-de201d8c273f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:41.035187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:41.053276 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24375003-def3-4960-9736-f0904b93d292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:41.053869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24375003-def3-4960-9736-f0904b93d292 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:41.054142 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24375003-def3-4960-9736-f0904b93d292 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:41.054408 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24375003-def3-4960-9736-f0904b93d292 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:41.055097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24375003-def3-4960-9736-f0904b93d292 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:41.055396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24375003-def3-4960-9736-f0904b93d292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:41.056740 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24375003-def3-4960-9736-f0904b93d292 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:41.064486 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 405/836] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:54:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:41.065600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ed57b-b2cf-42ec-9aef-a25002fa0f66 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:41.065874 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ed57b-b2cf-42ec-9aef-a25002fa0f66 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:41.066138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ed57b-b2cf-42ec-9aef-a25002fa0f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:41.084101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-b65ed57b-b2cf-42ec-9aef-a25002fa0f66 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:41.086495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b65ed57b-b2cf-42ec-9aef-a25002fa0f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:41.123046 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 432/837] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:54:40 2022] PUT /identity/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/0bd9c73ff2d94089a34de12c1155e1b5/roles/9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:54:41.123046 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f7cf1-719a-445a-9763-de201d8c273f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:41.195266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 406/838] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:54:41.235280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:41.262589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f7cf1-719a-445a-9763-de201d8c273f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:41.303392 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50449ee-6b54-43d3-9d5a-747eb603a0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:41.304022 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50449ee-6b54-43d3-9d5a-747eb603a0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:41.304295 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50449ee-6b54-43d3-9d5a-747eb603a0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:41.304557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50449ee-6b54-43d3-9d5a-747eb603a0d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:41.315271 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50449ee-6b54-43d3-9d5a-747eb603a0d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:41.316712 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50449ee-6b54-43d3-9d5a-747eb603a0d7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:41.332024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e50449ee-6b54-43d3-9d5a-747eb603a0d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:41.332024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:41.332024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:41.332024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:41.332024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:41.332024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:41.332024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:41.332024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:41.332024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:41.332024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:41.332024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:41.332024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:41.332024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:41.332024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:41.332024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:41.332024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:41.332985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:41.334485 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:41.334485 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:41.334485 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:41.334485 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 433/839] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:41.340885 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ed57b-b2cf-42ec-9aef-a25002fa0f66 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:41.443198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ed57b-b2cf-42ec-9aef-a25002fa0f66 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:41.458250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:41.480047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafcb319-f81c-4f9a-9b29-a8bcf8fdea65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:41.480663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcb319-f81c-4f9a-9b29-a8bcf8fdea65 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:41.480943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcb319-f81c-4f9a-9b29-a8bcf8fdea65 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:41.481213 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcb319-f81c-4f9a-9b29-a8bcf8fdea65 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:41.481862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcb319-f81c-4f9a-9b29-a8bcf8fdea65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:41.482161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcb319-f81c-4f9a-9b29-a8bcf8fdea65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:41.483603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcb319-f81c-4f9a-9b29-a8bcf8fdea65 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:41.486110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50449ee-6b54-43d3-9d5a-747eb603a0d7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:41.501900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 407/840] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:41.503407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafcb319-f81c-4f9a-9b29-a8bcf8fdea65 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:41.607777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50449ee-6b54-43d3-9d5a-747eb603a0d7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:41.621088 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafcb319-f81c-4f9a-9b29-a8bcf8fdea65 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:41.639905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:41.649703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8f9b6-1b54-4847-ae8e-bf5aee283f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:41.650327 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8f9b6-1b54-4847-ae8e-bf5aee283f5c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:41.654198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8f9b6-1b54-4847-ae8e-bf5aee283f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:41.654198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8f9b6-1b54-4847-ae8e-bf5aee283f5c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:41.654198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8f9b6-1b54-4847-ae8e-bf5aee283f5c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:41.654198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8f9b6-1b54-4847-ae8e-bf5aee283f5c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:41.660804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a5a9a-0a29-47df-9add-7ffa7977b25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:41.661985 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a5a9a-0a29-47df-9add-7ffa7977b25f None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:41.662260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a5a9a-0a29-47df-9add-7ffa7977b25f None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:41.662525 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a5a9a-0a29-47df-9add-7ffa7977b25f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c7b8f9b6-1b54-4847-ae8e-bf5aee283f5c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:41.664991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:41.666340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:41.666340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:41.666340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:41.666340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:41.666340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:41.666340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:41.666340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:41.666340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:41.666340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:41.666340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:41.666340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:41.666340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:41.666340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:41.666340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:41.666340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:41.666340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:41.667331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:41.667331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:41.667331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:41.667331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:41.667331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:41.667331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:41.667331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:41.667331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:41.667331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:41.667331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:41.667331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:41.667331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:41.667331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 07:54:41.667331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:41.668973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 434/841] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:54:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:41.673788 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8f9b6-1b54-4847-ae8e-bf5aee283f5c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:41.679821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a5a9a-0a29-47df-9add-7ffa7977b25f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:41.681819 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a5a9a-0a29-47df-9add-7ffa7977b25f None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:41.694449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-e60a5a9a-0a29-47df-9add-7ffa7977b25f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 06 07:54:41.694449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:41.694449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:41.694449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:41.694449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:41.694449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:41.694449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:41.694449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:41.694449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:41.694449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:41.694449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:41.694449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:41.694449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:41.694449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:41.694449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:41.694449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:41.695655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:41.696859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:41.696859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:41.696859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:41.696859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:41.696859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:41.696859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:41.696859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:41.696859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:41.696859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 06 07:54:41.696859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:41.698378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 408/842] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:54:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:41.707049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a5a9a-0a29-47df-9add-7ffa7977b25f None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:41.743896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:41.812381 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8f9b6-1b54-4847-ae8e-bf5aee283f5c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:41.827471 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a5a9a-0a29-47df-9add-7ffa7977b25f None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:41.860647 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677d3b8-b6a6-4494-9c27-8f028edd71e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:41.861496 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677d3b8-b6a6-4494-9c27-8f028edd71e7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:41.861921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677d3b8-b6a6-4494-9c27-8f028edd71e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:41.862462 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677d3b8-b6a6-4494-9c27-8f028edd71e7 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:41.863374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677d3b8-b6a6-4494-9c27-8f028edd71e7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:41.863821 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677d3b8-b6a6-4494-9c27-8f028edd71e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:41.864182 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3df3d-96af-41a6-b971-3a2c8c44fa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:41.865021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3df3d-96af-41a6-b971-3a2c8c44fa2a None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:41.865387 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3df3d-96af-41a6-b971-3a2c8c44fa2a None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:41.865748 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3df3d-96af-41a6-b971-3a2c8c44fa2a None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:41.866193 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677d3b8-b6a6-4494-9c27-8f028edd71e7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:41.866722 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3df3d-96af-41a6-b971-3a2c8c44fa2a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:41.867147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3df3d-96af-41a6-b971-3a2c8c44fa2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:41.868713 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3df3d-96af-41a6-b971-3a2c8c44fa2a None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:41.873472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 435/843] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:54:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:41.878645 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677d3b8-b6a6-4494-9c27-8f028edd71e7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:41.892809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 409/844] 10.222.0.30 () {56 vars in 1135 bytes} [Thu Oct 6 07:54:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:41.898683 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3df3d-96af-41a6-b971-3a2c8c44fa2a None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:41.966861 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:42.004736 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:42.021382 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3df3d-96af-41a6-b971-3a2c8c44fa2a None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:42.030155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677d3b8-b6a6-4494-9c27-8f028edd71e7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:42.058375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a7463-c890-49c8-920b-e7be5159cbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:42.059461 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a7463-c890-49c8-920b-e7be5159cbb0 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:42.059957 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a7463-c890-49c8-920b-e7be5159cbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:42.060471 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a7463-c890-49c8-920b-e7be5159cbb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:42.075155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f85d6-7cde-4baf-b6b2-b322c6de4295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:42.075761 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f85d6-7cde-4baf-b6b2-b322c6de4295 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:42.077567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f85d6-7cde-4baf-b6b2-b322c6de4295 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:42.077567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f85d6-7cde-4baf-b6b2-b322c6de4295 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:42.077567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f85d6-7cde-4baf-b6b2-b322c6de4295 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:42.077825 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f85d6-7cde-4baf-b6b2-b322c6de4295 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:42.079160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f85d6-7cde-4baf-b6b2-b322c6de4295 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:42.085310 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 436/845] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:54:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:42.086987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577528f4-9223-4f22-8e70-30d41b40d520 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:42.087274 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577528f4-9223-4f22-8e70-30d41b40d520 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:42.087548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577528f4-9223-4f22-8e70-30d41b40d520 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:42.089278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 437/846] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:42.097297 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b149a63-fe10-480a-a15e-82e3f5925fb0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:42.097750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b149a63-fe10-480a-a15e-82e3f5925fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:42.098252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b149a63-fe10-480a-a15e-82e3f5925fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:42.121882 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1b149a63-fe10-480a-a15e-82e3f5925fb0 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:42.123698 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1b149a63-fe10-480a-a15e-82e3f5925fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:42.165174 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a7463-c890-49c8-920b-e7be5159cbb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:42.167442 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a7463-c890-49c8-920b-e7be5159cbb0 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:42.228841 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 438/847] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:54:42.275805 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:42.293650 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 410/848] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 06 07:54:42.299026 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b149a63-fe10-480a-a15e-82e3f5925fb0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:42.355067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:42.409309 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b149a63-fe10-480a-a15e-82e3f5925fb0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:42.448511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3f28a-1007-4014-8d95-d4ba98586796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:42.449074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3f28a-1007-4014-8d95-d4ba98586796 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:42.449347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3f28a-1007-4014-8d95-d4ba98586796 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:42.449613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3f28a-1007-4014-8d95-d4ba98586796 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:42.461110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3f28a-1007-4014-8d95-d4ba98586796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:42.462610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3f28a-1007-4014-8d95-d4ba98586796 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:42.477962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-02f3f28a-1007-4014-8d95-d4ba98586796 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:42.477962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:42.477962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:42.477962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:42.477962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:42.477962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:42.477962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:42.477962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:42.477962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:42.477962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:42.477962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:42.477962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:42.477962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:42.477962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:42.477962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:42.477962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:42.479017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:42.480045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:42.480045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:42.480045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:42.480045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 439/849] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:42.485073 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a7463-c890-49c8-920b-e7be5159cbb0 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:42.506577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3f28a-1007-4014-8d95-d4ba98586796 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:42.615282 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a7463-c890-49c8-920b-e7be5159cbb0 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:42.618697 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3f28a-1007-4014-8d95-d4ba98586796 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:42.620508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:42.656961 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86212b-dc15-4180-80d2-5f1a857227d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:42.657691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86212b-dc15-4180-80d2-5f1a857227d3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:42.658029 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86212b-dc15-4180-80d2-5f1a857227d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:42.658368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86212b-dc15-4180-80d2-5f1a857227d3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:42.659056 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a86212b-dc15-4180-80d2-5f1a857227d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:42.660551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a86212b-dc15-4180-80d2-5f1a857227d3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:42.668528 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4dfe1-d1fd-4ba8-bec3-397a5633e98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:42.669983 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4dfe1-d1fd-4ba8-bec3-397a5633e98d None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:42.670472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4dfe1-d1fd-4ba8-bec3-397a5633e98d None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:42.670928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4dfe1-d1fd-4ba8-bec3-397a5633e98d None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:42.671772 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4dfe1-d1fd-4ba8-bec3-397a5633e98d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:42.672200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-7a86212b-dc15-4180-80d2-5f1a857227d3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 07:54:42.672200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:42.672200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:42.672200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:42.672200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:42.672200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:42.672200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:42.672200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:42.672200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 07:54:42.672200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:42.672200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:42.672200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:42.672200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:42.672200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:42.672200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:42.672200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:42.673296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:42.674394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:42.674394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:42.674394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:42.674394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:42.674394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:42.674394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:42.674394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:42.674394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:42.674394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:42.674394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:42.674394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:42.674394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:42.674394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 07:54:42.674394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:42.675374 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4dfe1-d1fd-4ba8-bec3-397a5633e98d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:42.676792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4dfe1-d1fd-4ba8-bec3-397a5633e98d None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:42.684663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 440/850] 10.222.0.30 () {56 vars in 1132 bytes} [Thu Oct 6 07:54:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:42.686556 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86212b-dc15-4180-80d2-5f1a857227d3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:42.703090 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 411/851] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:42.711577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4dfe1-d1fd-4ba8-bec3-397a5633e98d None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:42.759420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:42.811036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86212b-dc15-4180-80d2-5f1a857227d3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:42.821918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4dfe1-d1fd-4ba8-bec3-397a5633e98d None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:42.848753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b3c92-49a0-4a9e-a55f-d388b34a9f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:42.850144 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b3c92-49a0-4a9e-a55f-d388b34a9f6f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:42.850418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b3c92-49a0-4a9e-a55f-d388b34a9f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:42.850682 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b3c92-49a0-4a9e-a55f-d388b34a9f6f None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:42.851332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b3c92-49a0-4a9e-a55f-d388b34a9f6f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:42.851675 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b3c92-49a0-4a9e-a55f-d388b34a9f6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:42.853078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b3c92-49a0-4a9e-a55f-d388b34a9f6f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:42.862780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 441/852] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Oct 6 07:54:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:42.864779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b3c92-49a0-4a9e-a55f-d388b34a9f6f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:42.870647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00541434-c835-4291-a58a-a156af80cfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:42.871426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00541434-c835-4291-a58a-a156af80cfe7 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:42.871796 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00541434-c835-4291-a58a-a156af80cfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:42.872201 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00541434-c835-4291-a58a-a156af80cfe7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:42.879767 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00541434-c835-4291-a58a-a156af80cfe7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:42.881882 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00541434-c835-4291-a58a-a156af80cfe7 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:42.893631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-00541434-c835-4291-a58a-a156af80cfe7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:42.893631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:42.893631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:42.893631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:42.893631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:42.893631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:42.893631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:42.893631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:42.893631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:42.893631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:42.893631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:42.893631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:42.893631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:42.893631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:42.893631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:42.894730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:42.896444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:42.896444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:42.896444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:42.896444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:42.896444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:42.896444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:42.896444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:42.896444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:42.902636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 412/853] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:42.903651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00541434-c835-4291-a58a-a156af80cfe7 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:42.905621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:42.997605 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b3c92-49a0-4a9e-a55f-d388b34a9f6f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:43.013531 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00541434-c835-4291-a58a-a156af80cfe7 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:43.024652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:43.039563 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef5399-7ff6-4a14-b39d-c38a94d119f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:43.040748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef5399-7ff6-4a14-b39d-c38a94d119f6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:43.040748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef5399-7ff6-4a14-b39d-c38a94d119f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:43.040924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef5399-7ff6-4a14-b39d-c38a94d119f6 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:43.041667 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef5399-7ff6-4a14-b39d-c38a94d119f6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:43.042036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef5399-7ff6-4a14-b39d-c38a94d119f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:43.043506 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef5399-7ff6-4a14-b39d-c38a94d119f6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:43.049873 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 442/854] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Oct 6 07:54:42 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:43.065873 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924e27b-1e5b-4cb0-9640-3865eec612d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:43.066510 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924e27b-1e5b-4cb0-9640-3865eec612d6 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:43.066783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924e27b-1e5b-4cb0-9640-3865eec612d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:43.067140 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924e27b-1e5b-4cb0-9640-3865eec612d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:43.069024 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924e27b-1e5b-4cb0-9640-3865eec612d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:43.069328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924e27b-1e5b-4cb0-9640-3865eec612d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:43.070830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924e27b-1e5b-4cb0-9640-3865eec612d6 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:43.086846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 413/855] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:43.093091 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef5399-7ff6-4a14-b39d-c38a94d119f6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:43.210544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:43.217797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef5399-7ff6-4a14-b39d-c38a94d119f6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:43.257359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affbd0b7-1551-4bc7-a94f-a3673bc5b3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:43.257960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbd0b7-1551-4bc7-a94f-a3673bc5b3e5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:43.258232 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbd0b7-1551-4bc7-a94f-a3673bc5b3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:43.258500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbd0b7-1551-4bc7-a94f-a3673bc5b3e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:43.259344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbd0b7-1551-4bc7-a94f-a3673bc5b3e5 None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=3fa1c430c9f7416eaf1d17300b827e05) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:43.259607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbd0b7-1551-4bc7-a94f-a3673bc5b3e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:43.261133 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbd0b7-1551-4bc7-a94f-a3673bc5b3e5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:43.267665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 443/856] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:43 2022] GET /identity/v3/role_assignments?user.id=3fa1c430c9f7416eaf1d17300b827e05&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:43.429995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1e725-5b34-49ea-8c82-603fc9f4e2ec None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:43.430350 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1e725-5b34-49ea-8c82-603fc9f4e2ec None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:43.430540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1e725-5b34-49ea-8c82-603fc9f4e2ec None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:43.432164 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 414/857] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:43.440903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53cb76-155d-44aa-998a-8d4886718124 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:43.441411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53cb76-155d-44aa-998a-8d4886718124 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:43.441673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53cb76-155d-44aa-998a-8d4886718124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:43.461582 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-be53cb76-155d-44aa-998a-8d4886718124 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:43.462780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-be53cb76-155d-44aa-998a-8d4886718124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:43.534135 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 444/858] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:54:43.809942 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1e725-5b34-49ea-8c82-603fc9f4e2ec None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:43.922683 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1e725-5b34-49ea-8c82-603fc9f4e2ec None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:43.961299 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5bff7-c44f-45fd-b2ae-7749aff5ba40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:43.962316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5bff7-c44f-45fd-b2ae-7749aff5ba40 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:43.962801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5bff7-c44f-45fd-b2ae-7749aff5ba40 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:43.963304 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5bff7-c44f-45fd-b2ae-7749aff5ba40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:43.974376 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5bff7-c44f-45fd-b2ae-7749aff5ba40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:43.979539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-b2e5bff7-c44f-45fd-b2ae-7749aff5ba40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:43.979539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:43.979539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:43.979539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:43.979539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:43.979539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:43.979539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:43.979539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:43.979539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:43.979539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:43.979539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:43.979539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:43.979539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:43.979539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:43.979539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:43.979539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:43.980539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:43.980539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:43.980539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:43.980539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:43.980539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:43.980539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:43.980539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:43.980539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:43.980539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:43.980539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:43.980539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:43.980539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:43.980539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:43.980539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:43.980539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:43.980539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:43.981465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:43.981465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:43.981465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:43.981465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:43.981465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:43.981465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:43.981465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:43.981465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:43.981465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:43.981465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:43.981465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:54:43.981465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:43.984246 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 415/859] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:54:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:43.992807 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53cb76-155d-44aa-998a-8d4886718124 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:44.087071 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:44.098785 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53cb76-155d-44aa-998a-8d4886718124 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:44.134975 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23873658-6eb2-4c84-99c2-3dd61b965859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:44.135595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23873658-6eb2-4c84-99c2-3dd61b965859 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:44.135866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23873658-6eb2-4c84-99c2-3dd61b965859 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:44.136130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23873658-6eb2-4c84-99c2-3dd61b965859 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:44.136789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23873658-6eb2-4c84-99c2-3dd61b965859 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:44.137086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23873658-6eb2-4c84-99c2-3dd61b965859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:44.138439 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23873658-6eb2-4c84-99c2-3dd61b965859 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:44.148614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 445/860] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:54:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:44.156084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5bff7-c44f-45fd-b2ae-7749aff5ba40 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:44.183469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:44.260438 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5bff7-c44f-45fd-b2ae-7749aff5ba40 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:44.299362 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccedcc81-4941-4cbe-8f5e-df4f8500f9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:44.300011 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedcc81-4941-4cbe-8f5e-df4f8500f9ec None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:44.300283 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedcc81-4941-4cbe-8f5e-df4f8500f9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:44.300550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedcc81-4941-4cbe-8f5e-df4f8500f9ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:44.317055 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccedcc81-4941-4cbe-8f5e-df4f8500f9ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:44.319500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccedcc81-4941-4cbe-8f5e-df4f8500f9ec None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:44.334852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-ccedcc81-4941-4cbe-8f5e-df4f8500f9ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:44.334852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:44.334852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:44.334852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:44.334852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:44.334852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:44.334852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:44.334852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:44.334852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:44.334852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:44.334852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:44.334852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:44.334852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:44.334852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:44.334852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:44.334852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:44.335975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:44.336877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:44.336877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:44.336877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:44.338979 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 416/861] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:44.350962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23873658-6eb2-4c84-99c2-3dd61b965859 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:44.409693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecbe0d-2240-493f-a0a5-5630cab97fe6 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:44.410325 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecbe0d-2240-493f-a0a5-5630cab97fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:44.410818 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecbe0d-2240-493f-a0a5-5630cab97fe6 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:44.416061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 417/862] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:44.424877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddd9b9-4729-4eaf-9198-327060c18868 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:44.425796 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddd9b9-4729-4eaf-9198-327060c18868 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:44.426548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddd9b9-4729-4eaf-9198-327060c18868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:44.446840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-d2ddd9b9-4729-4eaf-9198-327060c18868 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:44.448549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d2ddd9b9-4729-4eaf-9198-327060c18868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:44.458766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23873658-6eb2-4c84-99c2-3dd61b965859 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:44.499399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dbbf0-b2a3-4fb4-80f6-f7c0d35f723e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:44.500138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dbbf0-b2a3-4fb4-80f6-f7c0d35f723e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:44.500468 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dbbf0-b2a3-4fb4-80f6-f7c0d35f723e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:44.500738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dbbf0-b2a3-4fb4-80f6-f7c0d35f723e None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:44.501433 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dbbf0-b2a3-4fb4-80f6-f7c0d35f723e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:44.501736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dbbf0-b2a3-4fb4-80f6-f7c0d35f723e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:44.503140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dbbf0-b2a3-4fb4-80f6-f7c0d35f723e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:44.518692 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 446/863] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:44.524778 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dbbf0-b2a3-4fb4-80f6-f7c0d35f723e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:44.535802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 418/864] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:54:44.536983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:44.626163 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dbbf0-b2a3-4fb4-80f6-f7c0d35f723e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:44.659990 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15b2f2-21c0-4e7d-980d-8b2853bf6393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:44.660664 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15b2f2-21c0-4e7d-980d-8b2853bf6393 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:44.660936 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15b2f2-21c0-4e7d-980d-8b2853bf6393 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:44.661794 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15b2f2-21c0-4e7d-980d-8b2853bf6393 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:44.669137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15b2f2-21c0-4e7d-980d-8b2853bf6393 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:44.670566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15b2f2-21c0-4e7d-980d-8b2853bf6393 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:44.682713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ea15b2f2-21c0-4e7d-980d-8b2853bf6393 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 07:54:44.682713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:44.682713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:44.682713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:44.682713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:44.682713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:44.682713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:44.682713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:44.682713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:44.682713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:44.682713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:44.682713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:44.682713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:44.682713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:44.682713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:44.682713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:44.683699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:44.685713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:44.685713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:44.685713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:44.685713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:44.685713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:44.685713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:44.685713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:44.685713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:44.685713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 07:54:44.685713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:44.685713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 447/865] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Oct 6 07:54:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:44.689737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddd9b9-4729-4eaf-9198-327060c18868 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:44.701706 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:44.789885 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddd9b9-4729-4eaf-9198-327060c18868 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:44.823968 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebd03e-15d6-438e-bddc-91d488db238e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:44.824585 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebd03e-15d6-438e-bddc-91d488db238e None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:44.824913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebd03e-15d6-438e-bddc-91d488db238e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:44.825101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebd03e-15d6-438e-bddc-91d488db238e None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:44.825793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcebd03e-15d6-438e-bddc-91d488db238e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:44.826089 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcebd03e-15d6-438e-bddc-91d488db238e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:44.827546 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcebd03e-15d6-438e-bddc-91d488db238e None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:44.845491 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 419/866] 10.222.0.30 () {56 vars in 1149 bytes} [Thu Oct 6 07:54:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:44.851221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15b2f2-21c0-4e7d-980d-8b2853bf6393 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:44.861912 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebd03e-15d6-438e-bddc-91d488db238e None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:44.923905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:44.947226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:44.956697 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15b2f2-21c0-4e7d-980d-8b2853bf6393 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:44.971464 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebd03e-15d6-438e-bddc-91d488db238e None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:44.992047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0f5b0-ab71-4fc6-af3a-52929331bb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:44.992734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0f5b0-ab71-4fc6-af3a-52929331bb72 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:44.992969 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0f5b0-ab71-4fc6-af3a-52929331bb72 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:44.993236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0f5b0-ab71-4fc6-af3a-52929331bb72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:45.011332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29915270-ed3c-4bfc-a6b2-922e518a22a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:45.012385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29915270-ed3c-4bfc-a6b2-922e518a22a3 None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:45.012893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29915270-ed3c-4bfc-a6b2-922e518a22a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:45.013417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29915270-ed3c-4bfc-a6b2-922e518a22a3 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:45.014296 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29915270-ed3c-4bfc-a6b2-922e518a22a3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:45.016032 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29915270-ed3c-4bfc-a6b2-922e518a22a3 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:45.064259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0f5b0-ab71-4fc6-af3a-52929331bb72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:45.065771 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0f5b0-ab71-4fc6-af3a-52929331bb72 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:45.079354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 420/867] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Oct 6 07:54:44 2022] POST /identity/v3/services => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:54:45.144740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:45.179557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 448/868] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:44 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 07:54:45.185292 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29915270-ed3c-4bfc-a6b2-922e518a22a3 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:45.316590 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29915270-ed3c-4bfc-a6b2-922e518a22a3 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:45.340838 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:45.353294 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7f20a-57bd-49ce-ac77-f39bcba56dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:45.355833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7f20a-57bd-49ce-ac77-f39bcba56dcc None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:45.355833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7f20a-57bd-49ce-ac77-f39bcba56dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:45.355833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7f20a-57bd-49ce-ac77-f39bcba56dcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:45.365864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7f20a-57bd-49ce-ac77-f39bcba56dcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:45.367409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7f20a-57bd-49ce-ac77-f39bcba56dcc None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:45.382652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-6ae7f20a-57bd-49ce-ac77-f39bcba56dcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:45.382652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:45.382652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:45.382652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:45.382652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:45.382652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:45.382652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:45.382652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:45.382652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:45.382652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:45.382652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:45.382652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:45.382652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:45.382652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:45.382652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:45.382652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:45.385482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:45.386379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:45.386379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:45.386379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:45.386379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 421/869] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:45.389390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0f5b0-ab71-4fc6-af3a-52929331bb72 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:45.416994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18051a5a-b8ab-4712-947f-dc13304f2bc3 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:45.417605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18051a5a-b8ab-4712-947f-dc13304f2bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:45.418074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18051a5a-b8ab-4712-947f-dc13304f2bc3 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:45.420176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 422/870] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:54:45.425616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc0984-aa77-486d-b86d-8f4a2626e542 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:45.426227 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc0984-aa77-486d-b86d-8f4a2626e542 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:45.426570 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc0984-aa77-486d-b86d-8f4a2626e542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:45.449231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-dcbc0984-aa77-486d-b86d-8f4a2626e542 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:45.450580 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-dcbc0984-aa77-486d-b86d-8f4a2626e542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:45.516443 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0f5b0-ab71-4fc6-af3a-52929331bb72 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:45.547395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 423/871] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:54:45.548915 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1feaf-3de4-482b-8765-968ef90bc6f7 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:45.549189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1feaf-3de4-482b-8765-968ef90bc6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:45.549455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1feaf-3de4-482b-8765-968ef90bc6f7 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:45.550804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 424/872] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:45.557391 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde141c-a891-4fca-aa80-f1f18acbe449 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:45.557667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde141c-a891-4fca-aa80-f1f18acbe449 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:45.557935 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde141c-a891-4fca-aa80-f1f18acbe449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:45.562752 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011de9f-b280-4aae-bf06-e790bd1126df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:45.563679 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011de9f-b280-4aae-bf06-e790bd1126df None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:45.564477 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011de9f-b280-4aae-bf06-e790bd1126df None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:45.565974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011de9f-b280-4aae-bf06-e790bd1126df None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:45.566854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011de9f-b280-4aae-bf06-e790bd1126df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:45.568028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011de9f-b280-4aae-bf06-e790bd1126df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:45.569820 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011de9f-b280-4aae-bf06-e790bd1126df None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:45.576892 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-4dde141c-a891-4fca-aa80-f1f18acbe449 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:45.578642 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4dde141c-a891-4fca-aa80-f1f18acbe449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:45.585291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 449/873] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:45.591006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011de9f-b280-4aae-bf06-e790bd1126df None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:45.627057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:45.651684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 425/874] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:54:45.703322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011de9f-b280-4aae-bf06-e790bd1126df None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:45.741540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e21aa6-edab-4312-b376-fe494da10a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:45.742455 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e21aa6-edab-4312-b376-fe494da10a8b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:45.742932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e21aa6-edab-4312-b376-fe494da10a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:45.743360 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e21aa6-edab-4312-b376-fe494da10a8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:45.751154 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e21aa6-edab-4312-b376-fe494da10a8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:45.752879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e21aa6-edab-4312-b376-fe494da10a8b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:45.765515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-73e21aa6-edab-4312-b376-fe494da10a8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:45.765515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:45.765515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:45.765515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:45.765515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:45.765515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:45.765515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:45.765515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:45.765515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:45.765515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:45.765515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:45.765515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:45.765515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:45.765515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:45.765515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:45.766631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:45.767869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:45.767869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:45.767869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:45.767869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:45.767869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:45.767869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:45.767869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:45.767869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:45.769581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 450/875] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:45.775946 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde141c-a891-4fca-aa80-f1f18acbe449 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:45.810815 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:45.825129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e21aa6-edab-4312-b376-fe494da10a8b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:45.885347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde141c-a891-4fca-aa80-f1f18acbe449 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:45.921792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1955c22-c4f4-491f-b689-efb691d665a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:45.922404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1955c22-c4f4-491f-b689-efb691d665a4 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:45.922677 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1955c22-c4f4-491f-b689-efb691d665a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:45.922963 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1955c22-c4f4-491f-b689-efb691d665a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:45.923836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1955c22-c4f4-491f-b689-efb691d665a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:45.924139 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1955c22-c4f4-491f-b689-efb691d665a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:45.925572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1955c22-c4f4-491f-b689-efb691d665a4 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:45.929078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:45.938453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e21aa6-edab-4312-b376-fe494da10a8b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:45.940039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 426/876] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:45.945077 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1955c22-c4f4-491f-b689-efb691d665a4 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:45.982506 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee40462-6f0c-4345-8941-4615f53633e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:45.983055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee40462-6f0c-4345-8941-4615f53633e8 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:45.983335 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee40462-6f0c-4345-8941-4615f53633e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:45.983606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee40462-6f0c-4345-8941-4615f53633e8 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:45.984279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee40462-6f0c-4345-8941-4615f53633e8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:45.985621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee40462-6f0c-4345-8941-4615f53633e8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:46.038151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:46.044530 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1955c22-c4f4-491f-b689-efb691d665a4 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:46.077795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c314840-e855-490f-bba2-13fd46263a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:46.078421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c314840-e855-490f-bba2-13fd46263a93 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:46.078697 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c314840-e855-490f-bba2-13fd46263a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:46.079129 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c314840-e855-490f-bba2-13fd46263a93 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:46.079607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c314840-e855-490f-bba2-13fd46263a93 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:46.083592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 451/877] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Oct 6 07:54:45 2022] POST /identity/v3/services => generated 269 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:54:46.083865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c314840-e855-490f-bba2-13fd46263a93 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:46.083971 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee40462-6f0c-4345-8941-4615f53633e8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-0c314840-e855-490f-bba2-13fd46263a93 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:46.095500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:46.096657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:46.096657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:46.096657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:46.096657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:46.096657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:46.096657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:46.096657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:46.096657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:46.096657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:46.096657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:46.096657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:46.096657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:46.096657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:46.096657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:46.096657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:46.096657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:46.098156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:46.098156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:46.098156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:46.098156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:46.098156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:46.098156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:46.098156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:46.098156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:46.098156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:46.098156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:46.098156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:46.098156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:46.098156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 07:54:46.098156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:46.098156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 427/878] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Oct 6 07:54:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:46.104157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c314840-e855-490f-bba2-13fd46263a93 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:46.236596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c314840-e855-490f-bba2-13fd46263a93 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:46.249877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee40462-6f0c-4345-8941-4615f53633e8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:46.276919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65ff80-662a-466e-9d63-859f43cf9131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:46.277540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65ff80-662a-466e-9d63-859f43cf9131 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:46.277916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65ff80-662a-466e-9d63-859f43cf9131 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:46.278098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65ff80-662a-466e-9d63-859f43cf9131 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:46.278706 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65ff80-662a-466e-9d63-859f43cf9131 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:46.279102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65ff80-662a-466e-9d63-859f43cf9131 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:46.280805 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65ff80-662a-466e-9d63-859f43cf9131 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:46.285543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 428/879] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Oct 6 07:54:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:46.292641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045f12c-6ace-4889-aa18-0b69ad5daf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:46.292817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65ff80-662a-466e-9d63-859f43cf9131 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:46.293226 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045f12c-6ace-4889-aa18-0b69ad5daf81 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:46.293502 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045f12c-6ace-4889-aa18-0b69ad5daf81 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:46.293769 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045f12c-6ace-4889-aa18-0b69ad5daf81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:46.294501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045f12c-6ace-4889-aa18-0b69ad5daf81 None admin] RBAC: Adding query filter params (role.id=9f0e36c2ed8843fbabd3a4cf30bc3725, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=0bd9c73ff2d94089a34de12c1155e1b5) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:46.294804 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045f12c-6ace-4889-aa18-0b69ad5daf81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:46.296258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045f12c-6ace-4889-aa18-0b69ad5daf81 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:46.302737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 452/880] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:46 2022] GET /identity/v3/role_assignments?user.id=0bd9c73ff2d94089a34de12c1155e1b5&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:46.321656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:46.410705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65ff80-662a-466e-9d63-859f43cf9131 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:46.450690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddbcd2-57e0-4fdc-9920-c218f9ff8029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:46.451353 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddbcd2-57e0-4fdc-9920-c218f9ff8029 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:46.451631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddbcd2-57e0-4fdc-9920-c218f9ff8029 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:46.451905 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddbcd2-57e0-4fdc-9920-c218f9ff8029 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:46.452638 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddbcd2-57e0-4fdc-9920-c218f9ff8029 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:46.452944 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddbcd2-57e0-4fdc-9920-c218f9ff8029 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:46.454373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddbcd2-57e0-4fdc-9920-c218f9ff8029 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:46.461755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 429/881] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Oct 6 07:54:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:47.522681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c62ce8-f60b-4a2f-93a1-c54bd1827a43 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:47.523261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c62ce8-f60b-4a2f-93a1-c54bd1827a43 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:47.523537 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c62ce8-f60b-4a2f-93a1-c54bd1827a43 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:47.525848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 453/882] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:54:47.532592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8d100-9bfd-40a5-a401-02c2b755eaa1 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:47.533023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8d100-9bfd-40a5-a401-02c2b755eaa1 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:47.533204 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8d100-9bfd-40a5-a401-02c2b755eaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:47.552704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-d7e8d100-9bfd-40a5-a401-02c2b755eaa1 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:47.553880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d7e8d100-9bfd-40a5-a401-02c2b755eaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:47.626706 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 430/883] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:54:47.900171 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c62ce8-f60b-4a2f-93a1-c54bd1827a43 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:47.913609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:48.007818 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c62ce8-f60b-4a2f-93a1-c54bd1827a43 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:48.044204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9fc10-de91-4dea-b8cc-db8638bee34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:48.044872 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9fc10-de91-4dea-b8cc-db8638bee34b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:48.045123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9fc10-de91-4dea-b8cc-db8638bee34b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:48.045389 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9fc10-de91-4dea-b8cc-db8638bee34b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:48.046000 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9fc10-de91-4dea-b8cc-db8638bee34b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:48.047390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9fc10-de91-4dea-b8cc-db8638bee34b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3ef9fc10-de91-4dea-b8cc-db8638bee34b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:48.056408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:48.057399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:48.057399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:48.057399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:48.057399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:48.057399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:48.057399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:48.057399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:48.057399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:48.057399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:48.057399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:48.057399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:48.057399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:48.057399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:48.057399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:48.057399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:48.057399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:48.058244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:48.058244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:48.058244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:48.058244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:48.058244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:48.058244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:48.058244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:48.058244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:48.058244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:48.058244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:48.058244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:48.058244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:48.058244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 07:54:48.058244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:48.058244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 454/884] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:54:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:48.065910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8d100-9bfd-40a5-a401-02c2b755eaa1 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:48.092812 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:48.185669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8d100-9bfd-40a5-a401-02c2b755eaa1 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:48.222925 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ed7c3-e23e-451b-a697-c74aee783474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:48.224610 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ed7c3-e23e-451b-a697-c74aee783474 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:48.225144 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ed7c3-e23e-451b-a697-c74aee783474 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:48.225621 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ed7c3-e23e-451b-a697-c74aee783474 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:48.226449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ed7c3-e23e-451b-a697-c74aee783474 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:48.226978 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ed7c3-e23e-451b-a697-c74aee783474 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:48.228551 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ed7c3-e23e-451b-a697-c74aee783474 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:48.234021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 431/885] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:54:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:48.240457 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9fc10-de91-4dea-b8cc-db8638bee34b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:48.324043 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:48.352649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9fc10-de91-4dea-b8cc-db8638bee34b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:48.389516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd60a3-7e64-4c0f-b58f-cda9110d256d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:48.390462 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd60a3-7e64-4c0f-b58f-cda9110d256d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:48.390989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd60a3-7e64-4c0f-b58f-cda9110d256d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:48.391453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd60a3-7e64-4c0f-b58f-cda9110d256d None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:48.392283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd60a3-7e64-4c0f-b58f-cda9110d256d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:48.392805 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd60a3-7e64-4c0f-b58f-cda9110d256d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:48.394380 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd60a3-7e64-4c0f-b58f-cda9110d256d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:48.400627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 455/886] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:54:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:48.407178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ed7c3-e23e-451b-a697-c74aee783474 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:48.434040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0830601-df39-44e7-8629-ca6274dbf315 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:48.434366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0830601-df39-44e7-8629-ca6274dbf315 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:48.434570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0830601-df39-44e7-8629-ca6274dbf315 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:48.436236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 456/887] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:48.443529 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914208ac-7f36-46e3-9744-940848db3a6e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:48.444004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914208ac-7f36-46e3-9744-940848db3a6e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:48.444422 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914208ac-7f36-46e3-9744-940848db3a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:48.465890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-914208ac-7f36-46e3-9744-940848db3a6e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:48.467809 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-914208ac-7f36-46e3-9744-940848db3a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:48.504132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:48.529260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ed7c3-e23e-451b-a697-c74aee783474 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:48.562975 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 457/888] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:54:48.579422 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f267c6-b785-4eaa-b975-6c1abcbe2fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:48.580070 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f267c6-b785-4eaa-b975-6c1abcbe2fba None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:48.580343 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f267c6-b785-4eaa-b975-6c1abcbe2fba None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:48.580608 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f267c6-b785-4eaa-b975-6c1abcbe2fba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:48.581355 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f267c6-b785-4eaa-b975-6c1abcbe2fba None admin] RBAC: Adding query filter params (interface=public, service_id=aaac718953264ae9922f6acb84defc1b, region_id=RegionOne) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:48.581747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f267c6-b785-4eaa-b975-6c1abcbe2fba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:48.583094 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f267c6-b785-4eaa-b975-6c1abcbe2fba None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:48.588749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 432/889] 10.222.0.30 () {56 vars in 1251 bytes} [Thu Oct 6 07:54:48 2022] GET /identity/v3/endpoints?service_id=aaac718953264ae9922f6acb84defc1b&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:48.594329 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914208ac-7f36-46e3-9744-940848db3a6e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:48.651392 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80c3f9-99ab-4ee0-9ff5-5d1fd2d191ca None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:48.651749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80c3f9-99ab-4ee0-9ff5-5d1fd2d191ca None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:48.651973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80c3f9-99ab-4ee0-9ff5-5d1fd2d191ca None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:48.653370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 433/890] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:48.659485 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65b9bf-4fbe-49c7-936f-f4b7f1f14c38 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:48.659711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65b9bf-4fbe-49c7-936f-f4b7f1f14c38 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:48.659923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65b9bf-4fbe-49c7-936f-f4b7f1f14c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:48.664862 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:48.682574 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-4d65b9bf-4fbe-49c7-936f-f4b7f1f14c38 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:48.683862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4d65b9bf-4fbe-49c7-936f-f4b7f1f14c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:48.703194 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914208ac-7f36-46e3-9744-940848db3a6e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:48.738721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedef2c-6f39-402b-880c-049e4264625d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:48.740169 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedef2c-6f39-402b-880c-049e4264625d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:48.740169 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedef2c-6f39-402b-880c-049e4264625d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:48.740457 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedef2c-6f39-402b-880c-049e4264625d None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:48.741018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedef2c-6f39-402b-880c-049e4264625d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:48.742373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedef2c-6f39-402b-880c-049e4264625d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:48.749391 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 458/891] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Oct 6 07:54:48 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 07:54:48.767119 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 434/892] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:54:48.791987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e05bf3-6e08-4527-a7ce-dbb87124fe53 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:48.792263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e05bf3-6e08-4527-a7ce-dbb87124fe53 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:48.792526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e05bf3-6e08-4527-a7ce-dbb87124fe53 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:48.794134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 459/893] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:48.799988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d541f1c-9772-4662-92e7-7a51312927e0 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:48.800290 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d541f1c-9772-4662-92e7-7a51312927e0 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:48.800527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d541f1c-9772-4662-92e7-7a51312927e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:48.820847 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-5d541f1c-9772-4662-92e7-7a51312927e0 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:48.822048 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5d541f1c-9772-4662-92e7-7a51312927e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:48.857932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e05bf3-6e08-4527-a7ce-dbb87124fe53 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:48.909229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 435/894] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:54:48.974617 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e05bf3-6e08-4527-a7ce-dbb87124fe53 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:49.006251 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:49.011440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ff0a4-6c38-40e9-90c7-145b8659e387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:49.012078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ff0a4-6c38-40e9-90c7-145b8659e387 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:49.012345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ff0a4-6c38-40e9-90c7-145b8659e387 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:49.012617 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ff0a4-6c38-40e9-90c7-145b8659e387 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:49.013207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ff0a4-6c38-40e9-90c7-145b8659e387 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:49.014562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ff0a4-6c38-40e9-90c7-145b8659e387 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:49.023442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c84ff0a4-6c38-40e9-90c7-145b8659e387 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 07:54:49.023442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:49.023442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:49.023442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:49.023442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:49.023442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:49.023442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:49.023442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:49.023442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 07:54:49.023442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:49.023442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:49.023442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:49.023442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:49.023442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:49.023442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:49.023442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:49.024383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:49.025442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:49.025442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:49.025442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:49.025442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:49.025442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:49.025442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:49.025442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:49.025442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:49.025442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:49.025442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:49.025442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:49.025442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:49.025442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 07:54:49.025442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:49.025442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 460/895] 10.222.0.30 () {56 vars in 1132 bytes} [Thu Oct 6 07:54:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:49.030580 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d541f1c-9772-4662-92e7-7a51312927e0 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:49.040242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:49.067902 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ff0a4-6c38-40e9-90c7-145b8659e387 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:49.156941 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d541f1c-9772-4662-92e7-7a51312927e0 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:49.201228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c10517-aaa0-4972-9738-958320537ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:49.201834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c10517-aaa0-4972-9738-958320537ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:49.202117 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c10517-aaa0-4972-9738-958320537ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:49.202384 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c10517-aaa0-4972-9738-958320537ff6 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:49.203051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c10517-aaa0-4972-9738-958320537ff6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:49.203360 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c10517-aaa0-4972-9738-958320537ff6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:49.204727 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c10517-aaa0-4972-9738-958320537ff6 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:49.207302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ff0a4-6c38-40e9-90c7-145b8659e387 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:49.210283 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 436/896] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Oct 6 07:54:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:49.211810 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c10517-aaa0-4972-9738-958320537ff6 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:49.245315 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933809c7-96c1-40e9-b157-efedfc1188cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:49.245940 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933809c7-96c1-40e9-b157-efedfc1188cc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:49.246218 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933809c7-96c1-40e9-b157-efedfc1188cc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:49.246487 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933809c7-96c1-40e9-b157-efedfc1188cc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:49.258626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933809c7-96c1-40e9-b157-efedfc1188cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:49.263983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-933809c7-96c1-40e9-b157-efedfc1188cc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 07:54:49.263983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:49.263983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:49.263983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:49.263983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:49.263983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:49.263983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:49.263983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:49.263983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:49.263983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:49.263983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:49.263983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:49.263983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:49.263983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:49.263983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:49.263983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:49.264923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:49.264923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:49.264923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:49.264923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:49.264923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:49.264923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:49.264923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:49.264923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:49.264923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:49.264923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:49.264923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:49.264923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:49.264923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:49.264923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:49.264923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:49.264923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:49.265797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:49.265797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:49.265797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:49.265797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:49.265797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:49.265797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:49.265797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:49.265797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:49.265797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:49.265797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:49.265797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 07:54:49.265797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:49.265797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 461/897] 10.222.0.30 () {56 vars in 1120 bytes} [Thu Oct 6 07:54:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:49.266538 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933809c7-96c1-40e9-b157-efedfc1188cc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:49.297670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:49.329819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:49.333889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c10517-aaa0-4972-9738-958320537ff6 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:49.368472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe710b-0cff-4143-8e74-7786670c6cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:49.377461 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe710b-0cff-4143-8e74-7786670c6cf1 None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:49.377461 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe710b-0cff-4143-8e74-7786670c6cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:49.377461 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe710b-0cff-4143-8e74-7786670c6cf1 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:49.377461 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe710b-0cff-4143-8e74-7786670c6cf1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:49.377461 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe710b-0cff-4143-8e74-7786670c6cf1 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:49.377906 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933809c7-96c1-40e9-b157-efedfc1188cc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:49.409444 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc2ae2-58d7-40bc-911a-3ba4ea3433a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:49.410240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc2ae2-58d7-40bc-911a-3ba4ea3433a6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:49.410713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc2ae2-58d7-40bc-911a-3ba4ea3433a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:49.411673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc2ae2-58d7-40bc-911a-3ba4ea3433a6 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:49.412366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc2ae2-58d7-40bc-911a-3ba4ea3433a6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:49.412665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc2ae2-58d7-40bc-911a-3ba4ea3433a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:49.414035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc2ae2-58d7-40bc-911a-3ba4ea3433a6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:49.418866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 462/898] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Oct 6 07:54:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:49.420088 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc2ae2-58d7-40bc-911a-3ba4ea3433a6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:49.480258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 437/899] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Oct 6 07:54:49 2022] POST /identity/v3/services => generated 241 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:54:49.481432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe710b-0cff-4143-8e74-7786670c6cf1 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:49.546615 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc2ae2-58d7-40bc-911a-3ba4ea3433a6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:49.576428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:49.605766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb9e68-427b-40a1-8e61-5be5bb0633d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:49.606666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe710b-0cff-4143-8e74-7786670c6cf1 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:49.607302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb9e68-427b-40a1-8e61-5be5bb0633d8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:49.607761 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb9e68-427b-40a1-8e61-5be5bb0633d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:49.608211 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb9e68-427b-40a1-8e61-5be5bb0633d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:49.609670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb9e68-427b-40a1-8e61-5be5bb0633d8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:49.611181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb9e68-427b-40a1-8e61-5be5bb0633d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:49.612964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb9e68-427b-40a1-8e61-5be5bb0633d8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:49.631679 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 463/900] 10.222.0.30 () {56 vars in 1115 bytes} [Thu Oct 6 07:54:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:49.633960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb9e68-427b-40a1-8e61-5be5bb0633d8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:49.648089 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:49.658841 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727c35b-8a0b-4e98-af71-73b5d54ac295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:49.670777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727c35b-8a0b-4e98-af71-73b5d54ac295 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:49.670777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727c35b-8a0b-4e98-af71-73b5d54ac295 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:49.670777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727c35b-8a0b-4e98-af71-73b5d54ac295 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:49.670777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727c35b-8a0b-4e98-af71-73b5d54ac295 None admin] RBAC: Adding query filter params (interface=public, service_id=b4f16e8368c8417b839ae024660efe73, region_id=RegionOne) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:49.670777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727c35b-8a0b-4e98-af71-73b5d54ac295 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:49.670777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727c35b-8a0b-4e98-af71-73b5d54ac295 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:49.671722 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 438/901] 10.222.0.30 () {56 vars in 1251 bytes} [Thu Oct 6 07:54:49 2022] GET /identity/v3/endpoints?service_id=b4f16e8368c8417b839ae024660efe73&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:49.678427 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727c35b-8a0b-4e98-af71-73b5d54ac295 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:49.760811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb9e68-427b-40a1-8e61-5be5bb0633d8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:49.795913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ee771-82f7-4b17-ace7-b8a38aa592a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:49.796537 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ee771-82f7-4b17-ace7-b8a38aa592a0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:49.796858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ee771-82f7-4b17-ace7-b8a38aa592a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:49.797070 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ee771-82f7-4b17-ace7-b8a38aa592a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:49.808185 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727c35b-8a0b-4e98-af71-73b5d54ac295 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:49.808389 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ee771-82f7-4b17-ace7-b8a38aa592a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:49.809683 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ee771-82f7-4b17-ace7-b8a38aa592a0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:49.825353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-609ee771-82f7-4b17-ace7-b8a38aa592a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:49.825353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:49.825353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:49.825353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:49.825353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:49.825353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:49.825353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:49.825353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:49.825353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:49.825353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:49.825353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:49.825353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:49.825353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:49.825353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:49.825353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:49.825353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:49.826355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:49.827869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:49.827869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:49.827869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:49.827869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 464/902] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:49.832361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ee771-82f7-4b17-ace7-b8a38aa592a0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:49.855116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1246c88-0267-4346-9aaf-260bb5046ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:49.856581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1246c88-0267-4346-9aaf-260bb5046ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:49.857343 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1246c88-0267-4346-9aaf-260bb5046ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:49.857845 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1246c88-0267-4346-9aaf-260bb5046ad7 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:49.858901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1246c88-0267-4346-9aaf-260bb5046ad7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:49.861174 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1246c88-0267-4346-9aaf-260bb5046ad7 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:49.865503 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:49.872365 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 439/903] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Oct 6 07:54:49 2022] GET /identity/v3/services => generated 1497 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 07:54:49.949020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ee771-82f7-4b17-ace7-b8a38aa592a0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:49.988345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a9254-4f6e-4082-bffa-985b1208fcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:49.989555 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a9254-4f6e-4082-bffa-985b1208fcac None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:49.989834 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a9254-4f6e-4082-bffa-985b1208fcac None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:49.990103 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a9254-4f6e-4082-bffa-985b1208fcac None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:49.990765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a9254-4f6e-4082-bffa-985b1208fcac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:49.991098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a9254-4f6e-4082-bffa-985b1208fcac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:49.992484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a9254-4f6e-4082-bffa-985b1208fcac None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:50.006087 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 465/904] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:50.011983 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1246c88-0267-4346-9aaf-260bb5046ad7 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:50.079243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:50.124416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1246c88-0267-4346-9aaf-260bb5046ad7 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:50.164517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1624bc-ae60-407d-be2b-0e0da5cd104f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:50.165378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1624bc-ae60-407d-be2b-0e0da5cd104f None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:50.165826 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1624bc-ae60-407d-be2b-0e0da5cd104f None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:50.166401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1624bc-ae60-407d-be2b-0e0da5cd104f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:50.174320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1624bc-ae60-407d-be2b-0e0da5cd104f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:50.176119 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1624bc-ae60-407d-be2b-0e0da5cd104f None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:50.187759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-5e1624bc-ae60-407d-be2b-0e0da5cd104f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 07:54:50.187759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:50.187759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:50.187759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:50.187759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:50.187759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:50.187759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:50.187759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:50.187759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:50.187759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:50.187759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:50.187759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:50.187759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:50.187759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:50.187759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:50.187759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:50.188728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:50.189664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:50.189664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:50.189664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:50.189664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:50.189664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:50.189664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:50.189664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:50.189664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:50.189664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 07:54:50.189664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:50.191796 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 440/905] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Oct 6 07:54:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:50.197789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a9254-4f6e-4082-bffa-985b1208fcac None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:50.299324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:50.305067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a9254-4f6e-4082-bffa-985b1208fcac None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:50.340981 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792aa2d3-140f-447c-8ce5-590111ce95ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:50.341509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792aa2d3-140f-447c-8ce5-590111ce95ff None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:50.341777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792aa2d3-140f-447c-8ce5-590111ce95ff None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:50.342045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792aa2d3-140f-447c-8ce5-590111ce95ff None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:50.342765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792aa2d3-140f-447c-8ce5-590111ce95ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:50.343097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792aa2d3-140f-447c-8ce5-590111ce95ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:50.344513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792aa2d3-140f-447c-8ce5-590111ce95ff None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:50.362901 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 466/906] 10.222.0.30 () {56 vars in 1149 bytes} [Thu Oct 6 07:54:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:50.368809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1624bc-ae60-407d-be2b-0e0da5cd104f None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:50.477919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1624bc-ae60-407d-be2b-0e0da5cd104f None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:50.495648 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:50.514899 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47552a12-23b4-4960-b677-7da590d7a537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:50.515535 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47552a12-23b4-4960-b677-7da590d7a537 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:50.515812 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47552a12-23b4-4960-b677-7da590d7a537 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:50.516086 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47552a12-23b4-4960-b677-7da590d7a537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:50.586758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47552a12-23b4-4960-b677-7da590d7a537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:50.588268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47552a12-23b4-4960-b677-7da590d7a537 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:50.695009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 441/907] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:54:50 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 07:54:50.700188 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792aa2d3-140f-447c-8ce5-590111ce95ff None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:50.805149 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792aa2d3-140f-447c-8ce5-590111ce95ff None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:50.840525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f29da4-7d18-4c78-a6c8-8160d3e1aa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:50.841154 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f29da4-7d18-4c78-a6c8-8160d3e1aa49 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:50.841426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f29da4-7d18-4c78-a6c8-8160d3e1aa49 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:50.841693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f29da4-7d18-4c78-a6c8-8160d3e1aa49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:50.852583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f29da4-7d18-4c78-a6c8-8160d3e1aa49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:50.854015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f29da4-7d18-4c78-a6c8-8160d3e1aa49 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:50.862644 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:50.869490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d1f29da4-7d18-4c78-a6c8-8160d3e1aa49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:50.869490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:50.869490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:50.869490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:50.869490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:50.869490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:50.869490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:50.869490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:50.869490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:50.869490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:50.869490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:50.869490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:50.869490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:50.869490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:50.869490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:50.869490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:50.870681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:50.871680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:50.871680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:50.871680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:50.872757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 467/908] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:50.878906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47552a12-23b4-4960-b677-7da590d7a537 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:50.898131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:50.993421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47552a12-23b4-4960-b677-7da590d7a537 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:51.029485 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3d06b-8f82-4784-bb35-dd1fc885dbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:51.030096 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3d06b-8f82-4784-bb35-dd1fc885dbc8 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:51.030377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3d06b-8f82-4784-bb35-dd1fc885dbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:51.030648 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3d06b-8f82-4784-bb35-dd1fc885dbc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:51.031314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3d06b-8f82-4784-bb35-dd1fc885dbc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:51.031614 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3d06b-8f82-4784-bb35-dd1fc885dbc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:51.033180 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3d06b-8f82-4784-bb35-dd1fc885dbc8 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:51.046024 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 442/909] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:51.051226 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f29da4-7d18-4c78-a6c8-8160d3e1aa49 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:51.064450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6652a3-3d0a-4816-b6da-cc89215fc12c None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:51.064736 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6652a3-3d0a-4816-b6da-cc89215fc12c None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:51.065013 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6652a3-3d0a-4816-b6da-cc89215fc12c None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:51.066592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 443/910] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:51.072657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377fe6d-977d-4cde-872a-1a1e8e67acfd None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:51.072944 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377fe6d-977d-4cde-872a-1a1e8e67acfd None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:51.073220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377fe6d-977d-4cde-872a-1a1e8e67acfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:51.092659 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-4377fe6d-977d-4cde-872a-1a1e8e67acfd None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:51.093845 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4377fe6d-977d-4cde-872a-1a1e8e67acfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:51.168012 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f29da4-7d18-4c78-a6c8-8160d3e1aa49 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:51.187924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 444/911] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:54:51.214145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64a352-9c59-4558-a5c8-57ac6176899e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:51.214773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64a352-9c59-4558-a5c8-57ac6176899e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:51.215078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64a352-9c59-4558-a5c8-57ac6176899e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:51.215349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64a352-9c59-4558-a5c8-57ac6176899e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:51.222188 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64a352-9c59-4558-a5c8-57ac6176899e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:51.224561 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64a352-9c59-4558-a5c8-57ac6176899e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:51.236562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1f64a352-9c59-4558-a5c8-57ac6176899e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:51.236562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:51.236562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:51.236562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:51.236562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:51.236562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:51.236562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:51.236562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:51.236562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:51.236562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:51.236562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:51.236562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:51.236562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:51.236562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:51.236562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:51.237549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:51.238466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:51.238466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:51.238466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:51.238466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:51.238466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:51.238466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:51.238466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:51.238466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:51.238466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 468/912] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:51.246038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377fe6d-977d-4cde-872a-1a1e8e67acfd None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:51.255927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:51.352537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377fe6d-977d-4cde-872a-1a1e8e67acfd None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:51.392490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8abcfc-f186-420b-9480-50d98f033b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:51.393113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8abcfc-f186-420b-9480-50d98f033b84 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:51.393379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8abcfc-f186-420b-9480-50d98f033b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:51.393649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8abcfc-f186-420b-9480-50d98f033b84 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:51.394361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8abcfc-f186-420b-9480-50d98f033b84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:51.394648 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8abcfc-f186-420b-9480-50d98f033b84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:51.396172 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8abcfc-f186-420b-9480-50d98f033b84 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:51.408221 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 445/913] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:51.414648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64a352-9c59-4558-a5c8-57ac6176899e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:51.476763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:51.494875 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8abcfc-f186-420b-9480-50d98f033b84 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:51.527551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64a352-9c59-4558-a5c8-57ac6176899e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:51.563845 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add68c1b-6bd4-4a73-9868-7601f1e415d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:51.564453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add68c1b-6bd4-4a73-9868-7601f1e415d6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:51.564725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add68c1b-6bd4-4a73-9868-7601f1e415d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:51.564994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add68c1b-6bd4-4a73-9868-7601f1e415d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:51.565875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add68c1b-6bd4-4a73-9868-7601f1e415d6 None admin] RBAC: Adding query filter params (role.id=c02e5654263d45f88dd97c056d1f9769, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=0bd9c73ff2d94089a34de12c1155e1b5) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:51.566229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add68c1b-6bd4-4a73-9868-7601f1e415d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:51.567953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add68c1b-6bd4-4a73-9868-7601f1e415d6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:51.574240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 469/914] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:54:51 2022] GET /identity/v3/role_assignments?user.id=0bd9c73ff2d94089a34de12c1155e1b5&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=c02e5654263d45f88dd97c056d1f9769 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:51.579964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:51.617446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8abcfc-f186-420b-9480-50d98f033b84 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:51.656344 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408091b-69ec-4ca8-9d86-6e3fa2272bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:51.656998 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408091b-69ec-4ca8-9d86-6e3fa2272bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:51.657284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408091b-69ec-4ca8-9d86-6e3fa2272bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:51.657555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408091b-69ec-4ca8-9d86-6e3fa2272bb7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:51.658156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408091b-69ec-4ca8-9d86-6e3fa2272bb7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:51.659607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408091b-69ec-4ca8-9d86-6e3fa2272bb7 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-9408091b-69ec-4ca8-9d86-6e3fa2272bb7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:51.670038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:51.671162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:51.671162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:51.671162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:51.671162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:51.671162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:51.671162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:51.671162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:51.671162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:51.671162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:51.671162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:51.671162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:51.671162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:51.671162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:51.671162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:51.671162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:51.671162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:51.672058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:51.672058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:51.672058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:51.672058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:51.672058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:51.672058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:51.672058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:51.672058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:51.672058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:51.672058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:51.672058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:51.672058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:51.672058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 06 07:54:51.672058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:51.672058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 446/915] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:54:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:51.679985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add68c1b-6bd4-4a73-9868-7601f1e415d6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:51.794419 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add68c1b-6bd4-4a73-9868-7601f1e415d6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:51.809353 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d96c5-2444-49b1-b692-ecb25422c8f7 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:51.809850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d96c5-2444-49b1-b692-ecb25422c8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:51.809928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d96c5-2444-49b1-b692-ecb25422c8f7 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:51.811418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 447/916] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:54:51.818618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301abd1-717a-47a5-be8b-297a3af5b79e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:51.818905 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301abd1-717a-47a5-be8b-297a3af5b79e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:51.819204 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301abd1-717a-47a5-be8b-297a3af5b79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:51.824435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:51.835678 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b759f5-dd4b-4a9f-9831-1ad216293437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:51.836566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b759f5-dd4b-4a9f-9831-1ad216293437 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:51.837008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b759f5-dd4b-4a9f-9831-1ad216293437 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:51.839953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b759f5-dd4b-4a9f-9831-1ad216293437 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:51.839953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b759f5-dd4b-4a9f-9831-1ad216293437 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:51.840130 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-2301abd1-717a-47a5-be8b-297a3af5b79e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:51.840206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b759f5-dd4b-4a9f-9831-1ad216293437 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:51.840275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-2301abd1-717a-47a5-be8b-297a3af5b79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:51.840502 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b759f5-dd4b-4a9f-9831-1ad216293437 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:51.845989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 470/917] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:54:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:51.852103 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b759f5-dd4b-4a9f-9831-1ad216293437 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:51.919466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 448/918] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:54:51.979842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b759f5-dd4b-4a9f-9831-1ad216293437 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:52.017567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8528bbd-faf1-43f2-8d0d-ed8766e195cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:52.018174 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8528bbd-faf1-43f2-8d0d-ed8766e195cc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:52.018453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8528bbd-faf1-43f2-8d0d-ed8766e195cc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:52.018723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8528bbd-faf1-43f2-8d0d-ed8766e195cc None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:52.019430 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8528bbd-faf1-43f2-8d0d-ed8766e195cc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:52.019728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8528bbd-faf1-43f2-8d0d-ed8766e195cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:52.021989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8528bbd-faf1-43f2-8d0d-ed8766e195cc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:52.027580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 471/919] 10.222.0.30 () {56 vars in 1114 bytes} [Thu Oct 6 07:54:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:52.033114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301abd1-717a-47a5-be8b-297a3af5b79e None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:52.094191 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:52.137255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301abd1-717a-47a5-be8b-297a3af5b79e None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:52.173400 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d4d31-9a5d-4346-bf54-c6c68c15d7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:52.174005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d4d31-9a5d-4346-bf54-c6c68c15d7f0 None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:52.174277 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d4d31-9a5d-4346-bf54-c6c68c15d7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:52.174543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d4d31-9a5d-4346-bf54-c6c68c15d7f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:52.175252 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d4d31-9a5d-4346-bf54-c6c68c15d7f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:52.177219 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d4d31-9a5d-4346-bf54-c6c68c15d7f0 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:52.248032 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8528bbd-faf1-43f2-8d0d-ed8766e195cc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:52.252909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 449/920] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 07:54:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:54:52.309375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930aabd-2085-4b8b-adab-e8f5338a8371 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:52.309742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930aabd-2085-4b8b-adab-e8f5338a8371 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:52.309960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930aabd-2085-4b8b-adab-e8f5338a8371 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:52.310661 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:52.312267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 450/921] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:54:52.318057 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2b487-fc41-4d12-8ea6-bfe614ef8498 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:52.318400 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2b487-fc41-4d12-8ea6-bfe614ef8498 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:52.318612 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2b487-fc41-4d12-8ea6-bfe614ef8498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:52.339093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-9dd2b487-fc41-4d12-8ea6-bfe614ef8498 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:52.340465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9dd2b487-fc41-4d12-8ea6-bfe614ef8498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:52.377917 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8528bbd-faf1-43f2-8d0d-ed8766e195cc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:52.422497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005cebb2-5b62-4ad7-8978-d44b80af4ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:52.423174 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005cebb2-5b62-4ad7-8978-d44b80af4ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:52.423450 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005cebb2-5b62-4ad7-8978-d44b80af4ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:52.423724 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005cebb2-5b62-4ad7-8978-d44b80af4ea7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:52.424593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005cebb2-5b62-4ad7-8978-d44b80af4ea7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:52.425762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005cebb2-5b62-4ad7-8978-d44b80af4ea7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:52.430988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 451/922] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-005cebb2-5b62-4ad7-8978-d44b80af4ea7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:52.439020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:52.440101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:52.440101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:52.440101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:52.440101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:52.440101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:52.440101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:52.440101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:52.440101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:52.440101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:52.440101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:52.440101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:52.440101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:52.440101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:52.440101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:52.440101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:52.440101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:52.440958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:52.440958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:52.440958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:52.440958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:52.440958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:52.440958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:52.440958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:52.440958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:52.440958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:52.440958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:52.440958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:52.440958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:52.440958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 07:54:52.440958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:52.440958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 472/923] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Oct 6 07:54:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:52.450285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2b487-fc41-4d12-8ea6-bfe614ef8498 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:52.519079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:52.560267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2b487-fc41-4d12-8ea6-bfe614ef8498 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:52.599081 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81a074-240e-4ecb-9e5e-d10eb2c50476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:52.600207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81a074-240e-4ecb-9e5e-d10eb2c50476 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:52.600633 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81a074-240e-4ecb-9e5e-d10eb2c50476 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:52.601053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81a074-240e-4ecb-9e5e-d10eb2c50476 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:52.601830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81a074-240e-4ecb-9e5e-d10eb2c50476 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:52.602296 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81a074-240e-4ecb-9e5e-d10eb2c50476 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:52.603872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81a074-240e-4ecb-9e5e-d10eb2c50476 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:52.609269 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 452/924] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Oct 6 07:54:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:52.615886 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005cebb2-5b62-4ad7-8978-d44b80af4ea7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:52.712643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81a074-240e-4ecb-9e5e-d10eb2c50476 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:52.732650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005cebb2-5b62-4ad7-8978-d44b80af4ea7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:52.750062 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:52.772904 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2004a-cc69-4b9b-b032-86aa5594cb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:52.773508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2004a-cc69-4b9b-b032-86aa5594cb87 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:52.773780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2004a-cc69-4b9b-b032-86aa5594cb87 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:52.774441 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2004a-cc69-4b9b-b032-86aa5594cb87 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:52.774750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2004a-cc69-4b9b-b032-86aa5594cb87 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:52.775082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2004a-cc69-4b9b-b032-86aa5594cb87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:52.776539 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2004a-cc69-4b9b-b032-86aa5594cb87 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:52.782403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 473/925] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Oct 6 07:54:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:52.788085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2004a-cc69-4b9b-b032-86aa5594cb87 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:52.860775 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81a074-240e-4ecb-9e5e-d10eb2c50476 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:52.876747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:52.895097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2004a-cc69-4b9b-b032-86aa5594cb87 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:52.912754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43594f-aa9a-4ab2-a3a5-44416260c498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:52.913618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43594f-aa9a-4ab2-a3a5-44416260c498 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:52.914003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43594f-aa9a-4ab2-a3a5-44416260c498 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:52.914369 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43594f-aa9a-4ab2-a3a5-44416260c498 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:52.915105 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43594f-aa9a-4ab2-a3a5-44416260c498 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:52.916572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43594f-aa9a-4ab2-a3a5-44416260c498 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:52.927533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-9a43594f-aa9a-4ab2-a3a5-44416260c498 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 07:54:52.927533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:52.927533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:52.927533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:52.927533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:52.927533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:52.927533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:52.927533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:52.927533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 07:54:52.927533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:52.927533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:52.927533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:52.927533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:52.927533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:52.927533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:52.927533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:52.928728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:52.929690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:52.929690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:52.929690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:52.929690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:52.929690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:52.929690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:52.929690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:52.929690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:52.929690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:52.929690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:52.929690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:52.929690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:52.929690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 06 07:54:52.929690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:52.930448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 453/926] 10.222.0.30 () {56 vars in 1132 bytes} [Thu Oct 6 07:54:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:52.935834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43594f-aa9a-4ab2-a3a5-44416260c498 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:52.943309 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b1fa7-d5fe-425f-8d93-61c679e958e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:52.944101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b1fa7-d5fe-425f-8d93-61c679e958e2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:52.944543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b1fa7-d5fe-425f-8d93-61c679e958e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:52.944973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b1fa7-d5fe-425f-8d93-61c679e958e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:52.945822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b1fa7-d5fe-425f-8d93-61c679e958e2 None admin] RBAC: Adding query filter params (interface=public, service_id=59cf113c9181439fbec41a642d7ed812, region_id=RegionOne) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:52.946373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b1fa7-d5fe-425f-8d93-61c679e958e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:52.948095 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b1fa7-d5fe-425f-8d93-61c679e958e2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:52.958326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 474/927] 10.222.0.30 () {56 vars in 1251 bytes} [Thu Oct 6 07:54:52 2022] GET /identity/v3/endpoints?service_id=59cf113c9181439fbec41a642d7ed812&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:52.963230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b1fa7-d5fe-425f-8d93-61c679e958e2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:53.016216 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:53.062543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43594f-aa9a-4ab2-a3a5-44416260c498 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:53.082391 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b1fa7-d5fe-425f-8d93-61c679e958e2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:53.104980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf920af-ca98-485a-817a-4a0eace81b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:53.105978 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf920af-ca98-485a-817a-4a0eace81b39 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:53.106529 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf920af-ca98-485a-817a-4a0eace81b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:53.107033 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf920af-ca98-485a-817a-4a0eace81b39 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:53.107862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf920af-ca98-485a-817a-4a0eace81b39 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:53.108361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf920af-ca98-485a-817a-4a0eace81b39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:53.109940 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf920af-ca98-485a-817a-4a0eace81b39 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:53.116620 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 454/928] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Oct 6 07:54:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:53.122032 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf920af-ca98-485a-817a-4a0eace81b39 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:53.123543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8e35c-edd0-477b-a536-5ddf381fce58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:53.124395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8e35c-edd0-477b-a536-5ddf381fce58 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:53.125010 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8e35c-edd0-477b-a536-5ddf381fce58 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:53.125480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8e35c-edd0-477b-a536-5ddf381fce58 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:53.126396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8e35c-edd0-477b-a536-5ddf381fce58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:53.127904 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8e35c-edd0-477b-a536-5ddf381fce58 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:53.138256 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 475/929] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Oct 6 07:54:52 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 07:54:53.170452 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:53.245124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf920af-ca98-485a-817a-4a0eace81b39 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:53.289559 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc50f1d-4a01-4e58-943a-9a8c183e71aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:53.290192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc50f1d-4a01-4e58-943a-9a8c183e71aa None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:53.290467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc50f1d-4a01-4e58-943a-9a8c183e71aa None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:53.290739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc50f1d-4a01-4e58-943a-9a8c183e71aa None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:53.291407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc50f1d-4a01-4e58-943a-9a8c183e71aa None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:53.291710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc50f1d-4a01-4e58-943a-9a8c183e71aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:53.293098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc50f1d-4a01-4e58-943a-9a8c183e71aa None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:53.299309 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 455/930] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Oct 6 07:54:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:53.308129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8e35c-edd0-477b-a536-5ddf381fce58 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:53.417578 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8e35c-edd0-477b-a536-5ddf381fce58 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:53.420379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:53.453363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ce450-3cea-4a07-91a7-5ff34e30e832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:53.454376 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ce450-3cea-4a07-91a7-5ff34e30e832 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:53.454881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ce450-3cea-4a07-91a7-5ff34e30e832 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:53.455410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ce450-3cea-4a07-91a7-5ff34e30e832 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:53.456279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ce450-3cea-4a07-91a7-5ff34e30e832 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:53.458406 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ce450-3cea-4a07-91a7-5ff34e30e832 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:53.559117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 476/931] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 07:54:53 2022] POST /identity/v3/endpoints => generated 347 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:54:53.819334 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c8ceb-56a0-4bb4-95d5-67260f0e531b None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:53.820001 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c8ceb-56a0-4bb4-95d5-67260f0e531b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:53.820487 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c8ceb-56a0-4bb4-95d5-67260f0e531b None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:53.822366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 456/932] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:53.827190 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb7941-ed41-4064-93d0-3fc09fcd406a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:53.827472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb7941-ed41-4064-93d0-3fc09fcd406a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:53.827741 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb7941-ed41-4064-93d0-3fc09fcd406a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:53.846345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-65eb7941-ed41-4064-93d0-3fc09fcd406a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:53.847553 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-65eb7941-ed41-4064-93d0-3fc09fcd406a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:53.925809 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 477/933] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:54:54.201705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c8ceb-56a0-4bb4-95d5-67260f0e531b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:54.322173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c8ceb-56a0-4bb4-95d5-67260f0e531b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:54.323735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:54.359546 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee4208-4788-4f57-8d44-cb220044c1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:54.360454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee4208-4788-4f57-8d44-cb220044c1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:54.360734 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee4208-4788-4f57-8d44-cb220044c1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:54.361009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee4208-4788-4f57-8d44-cb220044c1a8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:54.372342 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee4208-4788-4f57-8d44-cb220044c1a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:54.377407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-57ee4208-4788-4f57-8d44-cb220044c1a8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 07:54:54.377407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:54.377407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:54.377407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:54.377407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:54.377407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:54.377407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:54.377407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:54.377407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:54.377407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:54.377407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:54.377407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:54.377407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:54.377407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:54.377407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:54.377407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:54.378347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:54.378347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:54.378347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:54.378347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:54.378347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:54.378347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:54.378347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:54.378347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:54.378347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:54.378347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:54.378347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:54.378347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:54.378347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:54.378347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:54.378347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:54.378347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:54.379958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:54.379958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:54.379958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:54.379958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:54.379958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:54.379958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:54.379958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:54.379958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:54.379958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:54.379958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:54.379958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 07:54:54.379958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:54.379958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 457/934] 10.222.0.30 () {56 vars in 1120 bytes} [Thu Oct 6 07:54:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:54.385176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb7941-ed41-4064-93d0-3fc09fcd406a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:54.476807 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:54.493942 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb7941-ed41-4064-93d0-3fc09fcd406a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:54.531518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d38ba8-afdc-4a24-9d23-9dbc4c8a2e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:54.532617 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d38ba8-afdc-4a24-9d23-9dbc4c8a2e8d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:54.533170 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d38ba8-afdc-4a24-9d23-9dbc4c8a2e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:54.533618 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d38ba8-afdc-4a24-9d23-9dbc4c8a2e8d None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:54.534544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d38ba8-afdc-4a24-9d23-9dbc4c8a2e8d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:54.535067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d38ba8-afdc-4a24-9d23-9dbc4c8a2e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:54.536774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d38ba8-afdc-4a24-9d23-9dbc4c8a2e8d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:54.547711 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 478/935] 10.222.0.30 () {56 vars in 1115 bytes} [Thu Oct 6 07:54:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:54.558992 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee4208-4788-4f57-8d44-cb220044c1a8 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:54.659255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee4208-4788-4f57-8d44-cb220044c1a8 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:54.669762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4e783-ce6b-47fc-bf60-95684e95cd22 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:54.670336 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4e783-ce6b-47fc-bf60-95684e95cd22 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:54.670899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4e783-ce6b-47fc-bf60-95684e95cd22 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:54.673780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 479/936] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:54:54.677972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2d27b-32d4-4071-91a5-9af8b9df7edb None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:54.678379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2d27b-32d4-4071-91a5-9af8b9df7edb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:54.678758 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2d27b-32d4-4071-91a5-9af8b9df7edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:54.698638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c0e2d27b-32d4-4071-91a5-9af8b9df7edb None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:54.698955 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce632d7a-8e0b-4922-8f26-493f6ae10694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:54.699733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce632d7a-8e0b-4922-8f26-493f6ae10694 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:54.700149 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce632d7a-8e0b-4922-8f26-493f6ae10694 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:54.700280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c0e2d27b-32d4-4071-91a5-9af8b9df7edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:54.700688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce632d7a-8e0b-4922-8f26-493f6ae10694 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:54.713911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce632d7a-8e0b-4922-8f26-493f6ae10694 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:54.715594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce632d7a-8e0b-4922-8f26-493f6ae10694 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:54.738908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-ce632d7a-8e0b-4922-8f26-493f6ae10694 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:54.738908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:54.738908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:54.738908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:54.738908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:54.738908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:54.738908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:54.738908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:54.738908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:54.738908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:54.738908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:54.738908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:54.738908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:54.738908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:54.738908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:54.738908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:54.740102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:54.741015 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:54.741015 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:54.741015 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:54.742139 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 458/937] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:54.747864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce632d7a-8e0b-4922-8f26-493f6ae10694 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:54.765848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:54.784100 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 480/938] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:54:54.856083 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce632d7a-8e0b-4922-8f26-493f6ae10694 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:54.895617 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf49d1-505d-428b-8a6f-3c95268ef54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:54.896495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf49d1-505d-428b-8a6f-3c95268ef54c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:54.896920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf49d1-505d-428b-8a6f-3c95268ef54c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:54.897330 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf49d1-505d-428b-8a6f-3c95268ef54c None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:54.898107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf49d1-505d-428b-8a6f-3c95268ef54c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:54.898561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf49d1-505d-428b-8a6f-3c95268ef54c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:54.900097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf49d1-505d-428b-8a6f-3c95268ef54c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:54.913010 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 459/939] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:54.920250 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2d27b-32d4-4071-91a5-9af8b9df7edb None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:55.004036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:55.029318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2d27b-32d4-4071-91a5-9af8b9df7edb None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:55.056243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf49d1-505d-428b-8a6f-3c95268ef54c None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:55.065276 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f4cf9-c6a7-47c2-a46e-66416d60779a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:55.065908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f4cf9-c6a7-47c2-a46e-66416d60779a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:55.066197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f4cf9-c6a7-47c2-a46e-66416d60779a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:55.066620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f4cf9-c6a7-47c2-a46e-66416d60779a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:55.074721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f4cf9-c6a7-47c2-a46e-66416d60779a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:55.076183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f4cf9-c6a7-47c2-a46e-66416d60779a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:55.088973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-803f4cf9-c6a7-47c2-a46e-66416d60779a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 07:54:55.088973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:55.088973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:55.088973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:55.088973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:55.088973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:55.088973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:55.088973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:55.088973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:55.088973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:55.088973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:55.088973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:55.088973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:55.088973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:55.088973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:55.088973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:55.089956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:55.090897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:55.090897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:55.090897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:55.090897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:55.090897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:55.090897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:55.090897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:55.090897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:55.090897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 07:54:55.090897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:55.090897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 481/940] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Oct 6 07:54:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:55.095597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f4cf9-c6a7-47c2-a46e-66416d60779a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:55.172105 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:55.208568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf49d1-505d-428b-8a6f-3c95268ef54c None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:55.215950 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f4cf9-c6a7-47c2-a46e-66416d60779a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:55.254626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8bdc8-7910-4806-998f-de5032edb73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:55.255191 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8bdc8-7910-4806-998f-de5032edb73c None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:55.255469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8bdc8-7910-4806-998f-de5032edb73c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:55.255755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8bdc8-7910-4806-998f-de5032edb73c None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:55.256036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c9e41-849a-4a1a-b1bc-fea9fb615637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:55.256395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8bdc8-7910-4806-998f-de5032edb73c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:55.256968 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c9e41-849a-4a1a-b1bc-fea9fb615637 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:55.257386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c9e41-849a-4a1a-b1bc-fea9fb615637 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:55.257793 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c9e41-849a-4a1a-b1bc-fea9fb615637 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:55.257915 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8bdc8-7910-4806-998f-de5032edb73c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:55.258763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c9e41-849a-4a1a-b1bc-fea9fb615637 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:55.259258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c9e41-849a-4a1a-b1bc-fea9fb615637 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:55.260533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: WARNING keystone.server.flask.common [None req-14d8bdc8-7910-4806-998f-de5032edb73c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 06 07:54:55.260970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c9e41-849a-4a1a-b1bc-fea9fb615637 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:55.270920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:55.281528 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 482/941] 10.222.0.30 () {56 vars in 1149 bytes} [Thu Oct 6 07:54:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:55.288401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c9e41-849a-4a1a-b1bc-fea9fb615637 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:55.324903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.sql.core [None req-14d8bdc8-7910-4806-998f-de5032edb73c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 06 07:54:55.324903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 06 07:54:55.324903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [parameters: {'id': 'dcc6dfba22654de78b3c7f6c82025f16', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 06 07:54:55.324903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81479) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 06 07:54:55.348406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-14d8bdc8-7910-4806-998f-de5032edb73c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dcc6dfba22654de78b3c7f6c82025f16.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dcc6dfba22654de78b3c7f6c82025f16. Oct 06 07:54:55.348406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:55.348406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 07:54:55.348406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 07:54:55.348406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 07:54:55.348406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 07:54:55.348406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 07:54:55.348406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 07:54:55.348406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 07:54:55.348406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application conn.query(q) Oct 06 07:54:55.348406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 07:54:55.348406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 07:54:55.348406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application result.read() Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 06 07:54:55.349288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application next(self.gen) Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application yield resource Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application next(self.gen) Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self.session.rollback() Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self.force_reraise() Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise self.value Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application session.commit() Oct 06 07:54:55.350275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._prepare_impl() Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self.session.flush() Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._flush(objects) Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application compat.raise_( Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application flush_context.execute() Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 06 07:54:55.351622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rec.execute(self) Oct 06 07:54:55.352517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 06 07:54:55.352517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 06 07:54:55.352517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 06 07:54:55.352517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 06 07:54:55.352517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 06 07:54:55.352517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 06 07:54:55.352517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 06 07:54:55.352517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 06 07:54:55.352517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 06 07:54:55.352517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 06 07:54:55.352517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 06 07:54:55.352517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 06 07:54:55.352517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 06 07:54:55.352517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 06 07:54:55.352517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application conn.query(q) Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application result.read() Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 07:54:55.353310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 07:54:55.355098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 07:54:55.355098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 07:54:55.355098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 07:54:55.355098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 07:54:55.355098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 07:54:55.355098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 07:54:55.355098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 06 07:54:55.355098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 06 07:54:55.355098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [parameters: {'id': 'dcc6dfba22654de78b3c7f6c82025f16', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 06 07:54:55.355098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 06 07:54:55.355098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:55.355098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:55.355098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:55.355098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:55.355098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:55.356766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:55.358592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:55.358592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 06 07:54:55.358592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 06 07:54:55.358592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:55.358592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:55.358592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 06 07:54:55.358592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 06 07:54:55.358592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dcc6dfba22654de78b3c7f6c82025f16. Oct 06 07:54:55.358592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:55.358592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 460/942] 10.222.0.30 () {60 vars in 1140 bytes} [Thu Oct 6 07:54:55 2022] POST /identity/v3/projects => generated 262 bytes in 299 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:55.358592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8bdc8-7910-4806-998f-de5032edb73c None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:55.419649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c9e41-849a-4a1a-b1bc-fea9fb615637 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:55.460034 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74083d-2de7-40f7-8d32-28bac4e2e0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:55.461091 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74083d-2de7-40f7-8d32-28bac4e2e0b4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:55.461576 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74083d-2de7-40f7-8d32-28bac4e2e0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:55.462050 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74083d-2de7-40f7-8d32-28bac4e2e0b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:55.473684 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74083d-2de7-40f7-8d32-28bac4e2e0b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:55.475830 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74083d-2de7-40f7-8d32-28bac4e2e0b4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:55.493047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-bb74083d-2de7-40f7-8d32-28bac4e2e0b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:55.493047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:55.493047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:55.493047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:55.493047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:55.493047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:55.493047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:55.493047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:55.493047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:55.493047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:55.493047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:55.493047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:55.493047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:55.493047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:55.493047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:55.493047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:55.494020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:55.494958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:55.494958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:55.494958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:55.495740 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8bdc8-7910-4806-998f-de5032edb73c None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:55.497546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 483/943] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:55.503493 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74083d-2de7-40f7-8d32-28bac4e2e0b4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:55.533462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:55.540534 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71824b86-aba8-406c-9b3a-9453467e6655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:55.541167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71824b86-aba8-406c-9b3a-9453467e6655 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:55.541439 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71824b86-aba8-406c-9b3a-9453467e6655 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:55.542373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71824b86-aba8-406c-9b3a-9453467e6655 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:55.548514 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71824b86-aba8-406c-9b3a-9453467e6655 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:55.550144 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71824b86-aba8-406c-9b3a-9453467e6655 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:55.556796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:55.561322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-71824b86-aba8-406c-9b3a-9453467e6655 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:55.561322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:55.561322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:55.561322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:55.561322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:55.561322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:55.561322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:55.561322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:55.561322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:55.561322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:55.561322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:55.561322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:55.561322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:55.561322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:55.561322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:55.562229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:55.563155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:55.563155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:55.563155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:55.563155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:55.563155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:55.563155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:55.563155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:55.563155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:55.563155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 461/944] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:55.564095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763dbc2b-5ac1-4ed1-bf5d-f9ce729faf81 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:55.564095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763dbc2b-5ac1-4ed1-bf5d-f9ce729faf81 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:55.564228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763dbc2b-5ac1-4ed1-bf5d-f9ce729faf81 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:55.572395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 462/945] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:55.572395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763dbc2b-5ac1-4ed1-bf5d-f9ce729faf81 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:55.621834 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74083d-2de7-40f7-8d32-28bac4e2e0b4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:55.659199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf95a4c-998d-4ac8-8748-628e24c30532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:55.660222 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf95a4c-998d-4ac8-8748-628e24c30532 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:55.661171 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf95a4c-998d-4ac8-8748-628e24c30532 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:55.661686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf95a4c-998d-4ac8-8748-628e24c30532 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:55.662477 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf95a4c-998d-4ac8-8748-628e24c30532 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:55.662928 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf95a4c-998d-4ac8-8748-628e24c30532 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:55.664573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf95a4c-998d-4ac8-8748-628e24c30532 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:55.676995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763dbc2b-5ac1-4ed1-bf5d-f9ce729faf81 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:55.678639 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 484/946] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:55.679996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd423a2-1312-49e6-865b-7784cb030c56 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:55.680416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd423a2-1312-49e6-865b-7784cb030c56 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:55.680842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd423a2-1312-49e6-865b-7784cb030c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:55.698673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9cd423a2-1312-49e6-865b-7784cb030c56 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:55.699881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9cd423a2-1312-49e6-865b-7784cb030c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:55.716771 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4870d04-581a-4c63-bdb6-00d0521bac40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:55.717379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4870d04-581a-4c63-bdb6-00d0521bac40 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:55.717658 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4870d04-581a-4c63-bdb6-00d0521bac40 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:55.717923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4870d04-581a-4c63-bdb6-00d0521bac40 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:55.718597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4870d04-581a-4c63-bdb6-00d0521bac40 None admin] RBAC: Adding query filter params (name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:55.718879 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4870d04-581a-4c63-bdb6-00d0521bac40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:55.720348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4870d04-581a-4c63-bdb6-00d0521bac40 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:55.750970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 463/947] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Oct 6 07:54:55 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:55.750970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4870d04-581a-4c63-bdb6-00d0521bac40 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:55.801516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 485/948] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:54:55.819395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:55.869907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4870d04-581a-4c63-bdb6-00d0521bac40 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:55.914733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e90e84-0058-472a-b3d0-6f8d3e029e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:55.915434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e90e84-0058-472a-b3d0-6f8d3e029e7d None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:55.915711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e90e84-0058-472a-b3d0-6f8d3e029e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:55.915981 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e90e84-0058-472a-b3d0-6f8d3e029e7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:55.923836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e90e84-0058-472a-b3d0-6f8d3e029e7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:55.925321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e90e84-0058-472a-b3d0-6f8d3e029e7d None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:55.931373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21537235-f330-468f-a622-b8a563af13ee None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:55.931665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21537235-f330-468f-a622-b8a563af13ee None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:55.931912 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21537235-f330-468f-a622-b8a563af13ee None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:55.934671 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 486/949] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:54:55.938581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-26e90e84-0058-472a-b3d0-6f8d3e029e7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:55.938581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:55.938581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:55.938581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:55.938581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:55.938581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:55.938581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:55.938581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:55.938581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:55.938581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:55.938581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:55.938581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:55.938581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:54:55.938581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:54:55.938581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:55.939504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:55.940417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:54:55.940417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:54:55.940417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:54:55.940417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:54:55.940417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:54:55.940417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:54:55.940417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:54:55.940417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:55.940979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ba213-6f64-41bb-8da3-b9e6bf22605d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:55.940979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ba213-6f64-41bb-8da3-b9e6bf22605d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:55.940979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ba213-6f64-41bb-8da3-b9e6bf22605d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:55.941181 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 464/950] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:55.946878 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e90e84-0058-472a-b3d0-6f8d3e029e7d None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:55.961167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-518ba213-6f64-41bb-8da3-b9e6bf22605d None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:55.962349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-518ba213-6f64-41bb-8da3-b9e6bf22605d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:56.004116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:56.064554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 487/951] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:54:56.065742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e90e84-0058-472a-b3d0-6f8d3e029e7d None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:56.076926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:56.101652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57625525-fd77-4c5a-80d5-d7ebd4e8a5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:56.102284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57625525-fd77-4c5a-80d5-d7ebd4e8a5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:56.102557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57625525-fd77-4c5a-80d5-d7ebd4e8a5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:56.102821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57625525-fd77-4c5a-80d5-d7ebd4e8a5a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:56.103602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57625525-fd77-4c5a-80d5-d7ebd4e8a5a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:56.103904 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57625525-fd77-4c5a-80d5-d7ebd4e8a5a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:56.105438 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57625525-fd77-4c5a-80d5-d7ebd4e8a5a9 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:56.106354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ba213-6f64-41bb-8da3-b9e6bf22605d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:56.123509 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 465/952] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:54:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:56.129095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57625525-fd77-4c5a-80d5-d7ebd4e8a5a9 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:56.218017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ba213-6f64-41bb-8da3-b9e6bf22605d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:56.256193 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb403b6-4f06-4a18-8ff4-49d825d9b4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:56.256926 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb403b6-4f06-4a18-8ff4-49d825d9b4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:56.257272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb403b6-4f06-4a18-8ff4-49d825d9b4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:56.257608 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb403b6-4f06-4a18-8ff4-49d825d9b4f6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:56.257964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57625525-fd77-4c5a-80d5-d7ebd4e8a5a9 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:56.258406 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb403b6-4f06-4a18-8ff4-49d825d9b4f6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:56.259922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb403b6-4f06-4a18-8ff4-49d825d9b4f6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5fb403b6-4f06-4a18-8ff4-49d825d9b4f6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:56.269093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:56.270592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:56.270592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:56.270592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:56.270592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:56.270592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:56.270592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:56.270592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:56.270592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:56.270592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:56.270592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:56.270592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:56.270592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:56.270592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:56.270592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:56.270592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:56.270592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:56.271786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:56.271786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:56.271786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:56.271786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:56.271786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:56.271786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:56.271786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:56.271786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:56.271786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:56.271786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:56.271786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:56.271786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:56.271786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 07:54:56.271786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:56.278056 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 488/953] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Oct 6 07:54:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:56.280166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb403b6-4f06-4a18-8ff4-49d825d9b4f6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:56.308909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e39d1-aa7f-449d-adce-0f52c4e48c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:56.310084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e39d1-aa7f-449d-adce-0f52c4e48c58 None admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:56.310569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e39d1-aa7f-449d-adce-0f52c4e48c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:56.311268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e39d1-aa7f-449d-adce-0f52c4e48c58 None admin] PATH_INFO: `/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/0bd9c73ff2d94089a34de12c1155e1b5/roles/c02e5654263d45f88dd97c056d1f9769` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:56.314848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e39d1-aa7f-449d-adce-0f52c4e48c58 None admin] RBAC: Authorizing `identity:create_grant(project_id=399f0095cc8b4c64893ccc75af13c52d, user_id=0bd9c73ff2d94089a34de12c1155e1b5, role_id=c02e5654263d45f88dd97c056d1f9769)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:56.317983 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e39d1-aa7f-449d-adce-0f52c4e48c58 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:56.319294 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:56.391060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb403b6-4f06-4a18-8ff4-49d825d9b4f6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:56.405142 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 466/954] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:54:56 2022] PUT /identity/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/0bd9c73ff2d94089a34de12c1155e1b5/roles/c02e5654263d45f88dd97c056d1f9769 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:54:56.406229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e39d1-aa7f-449d-adce-0f52c4e48c58 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:56.446036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58f26b-8455-4fae-bef6-7b2c609c859a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:56.446664 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58f26b-8455-4fae-bef6-7b2c609c859a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:56.447041 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58f26b-8455-4fae-bef6-7b2c609c859a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:56.447696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58f26b-8455-4fae-bef6-7b2c609c859a None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:56.448036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58f26b-8455-4fae-bef6-7b2c609c859a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:56.448338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58f26b-8455-4fae-bef6-7b2c609c859a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:56.449793 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58f26b-8455-4fae-bef6-7b2c609c859a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:56.451071 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:56.456766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 489/955] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Oct 6 07:54:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:56.464308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58f26b-8455-4fae-bef6-7b2c609c859a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:56.554597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e39d1-aa7f-449d-adce-0f52c4e48c58 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:56.581879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:56.591017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58f26b-8455-4fae-bef6-7b2c609c859a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:56.605361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768dd9b-a7fb-40a4-a3a4-1e8d2aca6354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:56.606229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768dd9b-a7fb-40a4-a3a4-1e8d2aca6354 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:56.606638 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768dd9b-a7fb-40a4-a3a4-1e8d2aca6354 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:56.607127 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768dd9b-a7fb-40a4-a3a4-1e8d2aca6354 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:56.608182 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768dd9b-a7fb-40a4-a3a4-1e8d2aca6354 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:56.609090 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768dd9b-a7fb-40a4-a3a4-1e8d2aca6354 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-3768dd9b-a7fb-40a4-a3a4-1e8d2aca6354 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:56.619707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:56.620784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:56.620784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:56.620784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:56.620784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:56.620784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:56.620784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:56.620784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:56.620784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:56.620784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:56.620784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:56.620784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:56.620784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:56.620784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:56.620784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:56.620784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:56.620784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:56.622047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:56.622047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:56.622047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:56.622047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:56.622047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:56.622047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:56.622047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:56.622047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:56.622047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:56.622047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:56.622047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:56.622047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:56.622047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 07:54:56.622047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:56.622047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 467/956] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:54:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:56.626852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768dd9b-a7fb-40a4-a3a4-1e8d2aca6354 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:56.639856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8151a4c-fe77-4af3-bb99-e0a0eb90f95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:56.640794 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8151a4c-fe77-4af3-bb99-e0a0eb90f95f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:56.640909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8151a4c-fe77-4af3-bb99-e0a0eb90f95f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:56.641118 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8151a4c-fe77-4af3-bb99-e0a0eb90f95f None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:56.641753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8151a4c-fe77-4af3-bb99-e0a0eb90f95f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:56.642047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8151a4c-fe77-4af3-bb99-e0a0eb90f95f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:56.643570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8151a4c-fe77-4af3-bb99-e0a0eb90f95f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:56.654407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 490/957] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Oct 6 07:54:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:56.656474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8151a4c-fe77-4af3-bb99-e0a0eb90f95f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:56.744410 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:56.756393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768dd9b-a7fb-40a4-a3a4-1e8d2aca6354 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:56.791222 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8151a4c-fe77-4af3-bb99-e0a0eb90f95f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:56.805657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363787bf-4399-4b9e-9975-2475aa71cbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:56.806414 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363787bf-4399-4b9e-9975-2475aa71cbd1 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:56.806794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363787bf-4399-4b9e-9975-2475aa71cbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:56.807293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363787bf-4399-4b9e-9975-2475aa71cbd1 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:56.808045 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363787bf-4399-4b9e-9975-2475aa71cbd1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:56.808566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363787bf-4399-4b9e-9975-2475aa71cbd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:56.810023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363787bf-4399-4b9e-9975-2475aa71cbd1 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:56.816253 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 468/958] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Oct 6 07:54:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:56.822307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363787bf-4399-4b9e-9975-2475aa71cbd1 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:56.833059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68faaf10-b18a-4fe9-a183-b23cae83f49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:56.834155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faaf10-b18a-4fe9-a183-b23cae83f49b None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:56.834642 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faaf10-b18a-4fe9-a183-b23cae83f49b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:56.835145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faaf10-b18a-4fe9-a183-b23cae83f49b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:56.836030 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68faaf10-b18a-4fe9-a183-b23cae83f49b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:56.837746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68faaf10-b18a-4fe9-a183-b23cae83f49b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:56.927626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 491/959] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 07:54:56 2022] POST /identity/v3/endpoints => generated 349 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 07:54:56.938399 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363787bf-4399-4b9e-9975-2475aa71cbd1 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:56.986777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01951d-8334-4f45-b728-1b686d559736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:56.987514 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01951d-8334-4f45-b728-1b686d559736 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:56.987795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01951d-8334-4f45-b728-1b686d559736 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:56.988063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01951d-8334-4f45-b728-1b686d559736 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:56.988738 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01951d-8334-4f45-b728-1b686d559736 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:56.989592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01951d-8334-4f45-b728-1b686d559736 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:56.991156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01951d-8334-4f45-b728-1b686d559736 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:56.996879 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 469/960] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Oct 6 07:54:56 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:54:58.141446 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ddefa-735c-4689-92ce-7d12eb25b0a2 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:58.141986 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ddefa-735c-4689-92ce-7d12eb25b0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:58.141986 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ddefa-735c-4689-92ce-7d12eb25b0a2 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:58.143606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 492/961] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:58.149881 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c4ee5-32d0-4763-a902-dfc6659d91b2 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:58.150357 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c4ee5-32d0-4763-a902-dfc6659d91b2 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:58.150438 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c4ee5-32d0-4763-a902-dfc6659d91b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:58.169434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-4b8c4ee5-32d0-4763-a902-dfc6659d91b2 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:58.170998 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4b8c4ee5-32d0-4763-a902-dfc6659d91b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:58.186920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:58.248055 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 470/962] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:54:58.531417 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ddefa-735c-4689-92ce-7d12eb25b0a2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:58.551156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:58.650353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ddefa-735c-4689-92ce-7d12eb25b0a2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:58.685297 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa70b0-fe25-465f-aed1-86aef11f28b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:58.685907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa70b0-fe25-465f-aed1-86aef11f28b7 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:58.686184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa70b0-fe25-465f-aed1-86aef11f28b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:58.686458 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa70b0-fe25-465f-aed1-86aef11f28b7 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:58.687143 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa70b0-fe25-465f-aed1-86aef11f28b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:58.688583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa70b0-fe25-465f-aed1-86aef11f28b7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:58.690658 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717f920-0bea-4eb8-a279-48c55d703237 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:58.690958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717f920-0bea-4eb8-a279-48c55d703237 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:58.691241 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717f920-0bea-4eb8-a279-48c55d703237 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:58.691356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: WARNING keystone.server.flask.common [None req-4faa70b0-fe25-465f-aed1-86aef11f28b7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 06 07:54:58.693372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 471/963] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:58.698447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676135cf-9b30-4462-9a93-07aa97bc55ad None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:58.698721 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676135cf-9b30-4462-9a93-07aa97bc55ad None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:58.699021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676135cf-9b30-4462-9a93-07aa97bc55ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:58.717700 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-676135cf-9b30-4462-9a93-07aa97bc55ad None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:58.719209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-676135cf-9b30-4462-9a93-07aa97bc55ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:58.736014 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.sql.core [None req-4faa70b0-fe25-465f-aed1-86aef11f28b7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 06 07:54:58.736014 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 06 07:54:58.736014 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [parameters: {'user_id': '0a005ac0e1c644c78b62c103d1fb1668', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 06 07:54:58.736014 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81480) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 06 07:54:58.746723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-4faa70b0-fe25-465f-aed1-86aef11f28b7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 06 07:54:58.746723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:58.746723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 07:54:58.746723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 07:54:58.746723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 07:54:58.746723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 07:54:58.746723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 07:54:58.746723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 07:54:58.746723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 07:54:58.746723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application conn.query(q) Oct 06 07:54:58.746723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 07:54:58.746723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 07:54:58.746723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 07:54:58.746723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result.read() Oct 06 07:54:58.746723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 07:54:58.747665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application next(self.gen) Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application yield resource Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application next(self.gen) Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.session.rollback() Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.force_reraise() Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise self.value Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application session.commit() Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 06 07:54:58.748948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._prepare_impl() Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.session.flush() Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._flush(objects) Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application compat.raise_( Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application flush_context.execute() Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rec.execute(self) Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 06 07:54:58.749887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 06 07:54:58.750800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 06 07:54:58.750800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 06 07:54:58.750800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 06 07:54:58.750800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 06 07:54:58.750800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 06 07:54:58.750800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 06 07:54:58.750800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 06 07:54:58.750800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 06 07:54:58.750800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 06 07:54:58.750800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 06 07:54:58.750800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 06 07:54:58.750800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 06 07:54:58.750800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 06 07:54:58.750800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 06 07:54:58.750800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 06 07:54:58.750800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception Oct 06 07:54:58.751706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 06 07:54:58.751706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 06 07:54:58.751706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 06 07:54:58.751706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 06 07:54:58.751706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 06 07:54:58.751706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = self._query(query) Oct 06 07:54:58.751706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 06 07:54:58.751706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application conn.query(q) Oct 06 07:54:58.751706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 06 07:54:58.751706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 06 07:54:58.751706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 06 07:54:58.751706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result.read() Oct 06 07:54:58.751706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 06 07:54:58.751706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 06 07:54:58.751706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 06 07:54:58.751706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 06 07:54:58.754884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 06 07:54:58.754884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 06 07:54:58.754884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 06 07:54:58.754884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 06 07:54:58.754884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 06 07:54:58.754884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 06 07:54:58.754884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [parameters: {'user_id': '0a005ac0e1c644c78b62c103d1fb1668', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 06 07:54:58.754884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 06 07:54:58.754884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:58.754884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:58.754884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:58.754884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:58.754884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:58.754884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:58.754884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 06 07:54:58.758765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 06 07:54:58.761526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 06 07:54:58.761526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 06 07:54:58.761526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 06 07:54:58.761526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 06 07:54:58.761526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 06 07:54:58.761526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:58.761526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 493/964] 10.222.0.30 () {60 vars in 1131 bytes} [Thu Oct 6 07:54:58 2022] POST /identity/v3/users => generated 160 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:58.761526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa70b0-fe25-465f-aed1-86aef11f28b7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:58.809708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 472/965] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:54:58.853879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:58.876867 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa70b0-fe25-465f-aed1-86aef11f28b7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:58.913058 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527807f-b175-487d-9112-8736df95fd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:58.914038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527807f-b175-487d-9112-8736df95fd4f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:58.914540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527807f-b175-487d-9112-8736df95fd4f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:58.915036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527807f-b175-487d-9112-8736df95fd4f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:58.922732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e527807f-b175-487d-9112-8736df95fd4f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:58.924756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e527807f-b175-487d-9112-8736df95fd4f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:58.937388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e527807f-b175-487d-9112-8736df95fd4f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 07:54:58.937388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:58.937388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:58.937388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:58.937388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:58.937388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:58.937388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:58.937388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:58.937388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:58.937388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:58.937388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:58.937388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:58.937388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:54:58.937388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:54:58.937388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:54:58.937388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:58.938370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:58.939796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:58.939796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:58.939796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:54:58.939796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:54:58.939796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:54:58.939796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:54:58.939796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:54:58.939796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:54:58.939796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 06 07:54:58.939796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:58.942232 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 494/966] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:54:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:58.951212 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676135cf-9b30-4462-9a93-07aa97bc55ad None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:59.032158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:59.055034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676135cf-9b30-4462-9a93-07aa97bc55ad None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:59.087865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527807f-b175-487d-9112-8736df95fd4f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:59.093817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50f4c5-6fa2-4eb9-a703-bd7002cbfb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:59.094649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50f4c5-6fa2-4eb9-a703-bd7002cbfb71 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:59.095155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50f4c5-6fa2-4eb9-a703-bd7002cbfb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:59.095577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50f4c5-6fa2-4eb9-a703-bd7002cbfb71 None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:59.096516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50f4c5-6fa2-4eb9-a703-bd7002cbfb71 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:59.096996 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50f4c5-6fa2-4eb9-a703-bd7002cbfb71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:59.098622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50f4c5-6fa2-4eb9-a703-bd7002cbfb71 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:59.139510 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 473/967] 10.222.0.30 () {56 vars in 1101 bytes} [Thu Oct 6 07:54:58 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:59.216215 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527807f-b175-487d-9112-8736df95fd4f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:59.254755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab0809-b94f-4de2-9c14-f00b316a45f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:59.255423 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab0809-b94f-4de2-9c14-f00b316a45f7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:59.255694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab0809-b94f-4de2-9c14-f00b316a45f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:59.255957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab0809-b94f-4de2-9c14-f00b316a45f7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:59.264087 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:59.268976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab0809-b94f-4de2-9c14-f00b316a45f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:54:59.274395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c9ab0809-b94f-4de2-9c14-f00b316a45f7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 07:54:59.274395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:59.274395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:59.274395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:59.274395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:59.274395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:59.274395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:59.274395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:59.274395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:59.274395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:59.274395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:59.274395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:59.274395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:54:59.274395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:54:59.274395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:54:59.274395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:54:59.275452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:54:59.275452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:54:59.275452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:54:59.275452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:54:59.275452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:59.275452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:59.275452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:59.275452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:59.275452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:59.275452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:59.275452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:59.275452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:59.275452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:59.275452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:59.275452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:59.275452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:59.276321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:59.276321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:59.276321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:59.276321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:59.276321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:54:59.276321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:54:59.276321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:54:59.276321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:54:59.276321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:54:59.276321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:54:59.276321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 07:54:59.276321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:54:59.276321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 495/968] 10.222.0.30 () {56 vars in 1120 bytes} [Thu Oct 6 07:54:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:54:59.281939 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50f4c5-6fa2-4eb9-a703-bd7002cbfb71 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:59.302022 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d17e8-3b5e-4d7b-84b2-3614b3214d5f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:59.302255 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d17e8-3b5e-4d7b-84b2-3614b3214d5f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:59.302682 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d17e8-3b5e-4d7b-84b2-3614b3214d5f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:59.307949 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 496/969] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:54:59.314279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338266a-a1f0-4ce3-9e1d-ca4eca4c0cbe None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:59.314557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338266a-a1f0-4ce3-9e1d-ca4eca4c0cbe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:59.314826 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338266a-a1f0-4ce3-9e1d-ca4eca4c0cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:59.338667 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4338266a-a1f0-4ce3-9e1d-ca4eca4c0cbe None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:59.340703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4338266a-a1f0-4ce3-9e1d-ca4eca4c0cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:59.409972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50f4c5-6fa2-4eb9-a703-bd7002cbfb71 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:59.424615 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 497/970] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:54:59.425916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c39be-06ea-4e0f-b688-93d0fcf93efe None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:59.426811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c39be-06ea-4e0f-b688-93d0fcf93efe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:59.427118 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c39be-06ea-4e0f-b688-93d0fcf93efe None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:59.432747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 498/971] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:54:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:54:59.434210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04b125-594d-4da7-a24d-5c6ec66b2416 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:59.434489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04b125-594d-4da7-a24d-5c6ec66b2416 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:59.434764 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04b125-594d-4da7-a24d-5c6ec66b2416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:59.454545 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed0ea1-0ca1-4f1e-a495-ae069eea4428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:59.455225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed0ea1-0ca1-4f1e-a495-ae069eea4428 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:59.455499 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed0ea1-0ca1-4f1e-a495-ae069eea4428 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:59.455764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed0ea1-0ca1-4f1e-a495-ae069eea4428 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:59.456433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed0ea1-0ca1-4f1e-a495-ae069eea4428 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:59.456733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed0ea1-0ca1-4f1e-a495-ae069eea4428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:59.458043 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3b04b125-594d-4da7-a24d-5c6ec66b2416 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:54:59.458184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed0ea1-0ca1-4f1e-a495-ae069eea4428 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:59.459695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3b04b125-594d-4da7-a24d-5c6ec66b2416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:54:59.460733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:59.467708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 474/972] 10.222.0.30 () {56 vars in 1115 bytes} [Thu Oct 6 07:54:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:59.474049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed0ea1-0ca1-4f1e-a495-ae069eea4428 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:59.530614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 499/973] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:54:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:54:59.589661 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed0ea1-0ca1-4f1e-a495-ae069eea4428 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:59.624288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64641144-bebb-4c10-8291-89f4c826c9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:59.624916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64641144-bebb-4c10-8291-89f4c826c9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:59.625219 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64641144-bebb-4c10-8291-89f4c826c9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:59.625490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64641144-bebb-4c10-8291-89f4c826c9a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:59.637523 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64641144-bebb-4c10-8291-89f4c826c9a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:59.639173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64641144-bebb-4c10-8291-89f4c826c9a2 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:59.655173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-64641144-bebb-4c10-8291-89f4c826c9a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:59.655173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:59.655173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:59.655173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:59.655173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:59.655173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:59.655173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:59.655173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:59.655173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:59.655173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:59.655173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:59.655173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:59.655173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:54:59.655173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:54:59.655173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:54:59.655173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:59.656177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:54:59.657107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:54:59.657107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:54:59.657107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:59.659257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 475/974] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:59.662060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04b125-594d-4da7-a24d-5c6ec66b2416 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:59.693329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64641144-bebb-4c10-8291-89f4c826c9a2 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:59.725704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:59.745816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:59.775236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04b125-594d-4da7-a24d-5c6ec66b2416 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:59.797861 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64641144-bebb-4c10-8291-89f4c826c9a2 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:59.810892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a2299-c4cd-42aa-922a-2010168a2c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:59.811625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a2299-c4cd-42aa-922a-2010168a2c4a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:59.811898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a2299-c4cd-42aa-922a-2010168a2c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:59.812286 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a2299-c4cd-42aa-922a-2010168a2c4a None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:59.812923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a2299-c4cd-42aa-922a-2010168a2c4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:54:59.813220 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a2299-c4cd-42aa-922a-2010168a2c4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:59.814643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a2299-c4cd-42aa-922a-2010168a2c4a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:59.827375 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 500/975] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:54:59.828168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a2299-c4cd-42aa-922a-2010168a2c4a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:59.838236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cc766-71b5-4301-ac86-57b6af8a0bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:59.838866 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cc766-71b5-4301-ac86-57b6af8a0bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:59.839175 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cc766-71b5-4301-ac86-57b6af8a0bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:59.839448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cc766-71b5-4301-ac86-57b6af8a0bb7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:59.840085 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36cc766-71b5-4301-ac86-57b6af8a0bb7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:59.841433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36cc766-71b5-4301-ac86-57b6af8a0bb7 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-b36cc766-71b5-4301-ac86-57b6af8a0bb7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:54:59.850366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:54:59.851445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:54:59.851445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:54:59.851445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:54:59.851445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:54:59.851445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:54:59.851445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:54:59.851445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:54:59.851445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:54:59.851445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:54:59.851445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:54:59.851445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:54:59.851445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:54:59.851445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:54:59.851445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:54:59.851445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:54:59.851445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:54:59.852444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:54:59.852444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:54:59.852444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:54:59.852444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:54:59.852444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:54:59.852444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:54:59.852444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:54:59.852444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:54:59.852444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:54:59.852444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:54:59.852444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:54:59.852444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:54:59.852444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 07:54:59.852444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:54:59.852444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 476/976] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Oct 6 07:54:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:54:59.853272 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cc766-71b5-4301-ac86-57b6af8a0bb7 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:54:59.929090 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a2299-c4cd-42aa-922a-2010168a2c4a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:59.946504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:59.952116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cc766-71b5-4301-ac86-57b6af8a0bb7 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:54:59.963506 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643a7c7-60b6-4b05-a938-0055b9d1ab48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:59.964108 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643a7c7-60b6-4b05-a938-0055b9d1ab48 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:59.964384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643a7c7-60b6-4b05-a938-0055b9d1ab48 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:59.964653 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643a7c7-60b6-4b05-a938-0055b9d1ab48 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:59.965305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643a7c7-60b6-4b05-a938-0055b9d1ab48 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:59.965989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:54:59.966663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643a7c7-60b6-4b05-a938-0055b9d1ab48 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:54:59.985810 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302f63b-8cf2-4cea-b98a-4e0bf27c4005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:54:59.986430 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302f63b-8cf2-4cea-b98a-4e0bf27c4005 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:54:59.986893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302f63b-8cf2-4cea-b98a-4e0bf27c4005 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:54:59.987132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302f63b-8cf2-4cea-b98a-4e0bf27c4005 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:54:59.994349 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302f63b-8cf2-4cea-b98a-4e0bf27c4005 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:54:59.995819 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302f63b-8cf2-4cea-b98a-4e0bf27c4005 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:00.007803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-7302f63b-8cf2-4cea-b98a-4e0bf27c4005 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 07:55:00.007803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:00.007803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:00.007803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:00.007803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:00.007803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:00.007803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:00.007803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:00.007803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:00.007803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:00.007803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:00.007803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:00.007803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:55:00.007803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:55:00.007803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:55:00.007803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:00.013592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:00.018250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:00.018250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:00.018250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:55:00.018250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:55:00.018250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:55:00.018250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:55:00.018250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:55:00.018250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:55:00.018250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 06 07:55:00.018250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:00.018250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 477/977] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Oct 6 07:54:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:00.018250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302f63b-8cf2-4cea-b98a-4e0bf27c4005 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:00.105751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302f63b-8cf2-4cea-b98a-4e0bf27c4005 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:00.139465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccbdc9-d7b4-4a2f-9431-da8a4a0f5217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:00.140192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccbdc9-d7b4-4a2f-9431-da8a4a0f5217 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:00.140487 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccbdc9-d7b4-4a2f-9431-da8a4a0f5217 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:00.140744 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccbdc9-d7b4-4a2f-9431-da8a4a0f5217 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:00.141367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccbdc9-d7b4-4a2f-9431-da8a4a0f5217 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:00.141675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccbdc9-d7b4-4a2f-9431-da8a4a0f5217 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:00.147075 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 501/978] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Oct 6 07:54:59 2022] POST /identity/v3/services => generated 242 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:55:00.147075 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643a7c7-60b6-4b05-a938-0055b9d1ab48 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:00.147351 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccbdc9-d7b4-4a2f-9431-da8a4a0f5217 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:00.150994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 478/979] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Oct 6 07:55:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:00.157019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccbdc9-d7b4-4a2f-9431-da8a4a0f5217 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:00.207235 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:00.279095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccbdc9-d7b4-4a2f-9431-da8a4a0f5217 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:00.287894 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643a7c7-60b6-4b05-a938-0055b9d1ab48 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:00.320191 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b43f9a-5675-4f85-88fc-80e4d4d82660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:00.320798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b43f9a-5675-4f85-88fc-80e4d4d82660 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:00.321074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b43f9a-5675-4f85-88fc-80e4d4d82660 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:00.321339 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b43f9a-5675-4f85-88fc-80e4d4d82660 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:00.321974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b43f9a-5675-4f85-88fc-80e4d4d82660 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:00.322274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b43f9a-5675-4f85-88fc-80e4d4d82660 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:00.323675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b43f9a-5675-4f85-88fc-80e4d4d82660 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:00.328505 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 479/980] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Oct 6 07:55:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:00.329465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e46f65-360f-4ff6-8fdd-4fc9c62426eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:00.330136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e46f65-360f-4ff6-8fdd-4fc9c62426eb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:00.330416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e46f65-360f-4ff6-8fdd-4fc9c62426eb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:00.330684 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e46f65-360f-4ff6-8fdd-4fc9c62426eb None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:00.331437 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e46f65-360f-4ff6-8fdd-4fc9c62426eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:00.331744 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e46f65-360f-4ff6-8fdd-4fc9c62426eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:00.333925 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e46f65-360f-4ff6-8fdd-4fc9c62426eb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:00.335255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b43f9a-5675-4f85-88fc-80e4d4d82660 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:00.354209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 502/981] 10.222.0.30 () {56 vars in 1149 bytes} [Thu Oct 6 07:55:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:00.359514 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e46f65-360f-4ff6-8fdd-4fc9c62426eb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:00.392769 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:00.447747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b43f9a-5675-4f85-88fc-80e4d4d82660 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:00.450982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:00.467157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e46f65-360f-4ff6-8fdd-4fc9c62426eb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:00.483428 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf589c3-13ce-4ee2-a1d2-11b74312ab61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:00.484215 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf589c3-13ce-4ee2-a1d2-11b74312ab61 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:00.484650 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf589c3-13ce-4ee2-a1d2-11b74312ab61 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:00.484983 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf589c3-13ce-4ee2-a1d2-11b74312ab61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:00.485957 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf589c3-13ce-4ee2-a1d2-11b74312ab61 None admin] RBAC: Adding query filter params (interface=admin, service_id=59cf113c9181439fbec41a642d7ed812, region_id=RegionOne) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:00.486270 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf589c3-13ce-4ee2-a1d2-11b74312ab61 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:00.487682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf589c3-13ce-4ee2-a1d2-11b74312ab61 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:00.493552 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 480/982] 10.222.0.30 () {56 vars in 1249 bytes} [Thu Oct 6 07:55:00 2022] GET /identity/v3/endpoints?service_id=59cf113c9181439fbec41a642d7ed812&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:00.499296 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf589c3-13ce-4ee2-a1d2-11b74312ab61 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:00.507996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b3152-5836-4bba-a2ab-3ccf997cfd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:00.508858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b3152-5836-4bba-a2ab-3ccf997cfd57 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:00.509410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b3152-5836-4bba-a2ab-3ccf997cfd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:00.509754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b3152-5836-4bba-a2ab-3ccf997cfd57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:00.583253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b3152-5836-4bba-a2ab-3ccf997cfd57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:00.584697 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b3152-5836-4bba-a2ab-3ccf997cfd57 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:00.606180 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf589c3-13ce-4ee2-a1d2-11b74312ab61 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:00.620243 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:00.641783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0e7f3-4c38-4aa0-8be5-b19aa19684f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:00.642387 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0e7f3-4c38-4aa0-8be5-b19aa19684f6 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:00.642669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0e7f3-4c38-4aa0-8be5-b19aa19684f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:00.643007 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0e7f3-4c38-4aa0-8be5-b19aa19684f6 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:00.643591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0e7f3-4c38-4aa0-8be5-b19aa19684f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:00.645090 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0e7f3-4c38-4aa0-8be5-b19aa19684f6 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:00.652011 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 481/983] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Oct 6 07:55:00 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 07:55:00.702837 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 503/984] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:55:00 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 06 07:55:00.708740 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0e7f3-4c38-4aa0-8be5-b19aa19684f6 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:00.743362 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:00.823223 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0e7f3-4c38-4aa0-8be5-b19aa19684f6 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:00.860113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61548275-a86f-442c-9162-e36484a5b4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:00.860746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61548275-a86f-442c-9162-e36484a5b4cf None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:00.861024 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61548275-a86f-442c-9162-e36484a5b4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:00.861293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61548275-a86f-442c-9162-e36484a5b4cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:00.873596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61548275-a86f-442c-9162-e36484a5b4cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:00.875009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61548275-a86f-442c-9162-e36484a5b4cf None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:00.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-61548275-a86f-442c-9162-e36484a5b4cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:55:00.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:00.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:00.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:00.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:00.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:00.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:00.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:00.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:00.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:00.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:00.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:00.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:55:00.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:55:00.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:55:00.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:00.891715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:55:00.892663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:55:00.892663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:55:00.892663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:00.892663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 482/985] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:00.900200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b3152-5836-4bba-a2ab-3ccf997cfd57 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:01.004353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b3152-5836-4bba-a2ab-3ccf997cfd57 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:01.032966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:01.038992 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f4a62-0bc6-4bff-93c0-6ecf7229ad55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:01.039607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f4a62-0bc6-4bff-93c0-6ecf7229ad55 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:01.039885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f4a62-0bc6-4bff-93c0-6ecf7229ad55 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:01.040157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f4a62-0bc6-4bff-93c0-6ecf7229ad55 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:01.040805 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f4a62-0bc6-4bff-93c0-6ecf7229ad55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:01.041106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f4a62-0bc6-4bff-93c0-6ecf7229ad55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:01.042468 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f4a62-0bc6-4bff-93c0-6ecf7229ad55 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:01.061195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 504/986] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:01.063383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61548275-a86f-442c-9162-e36484a5b4cf None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:01.152633 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:01.171370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61548275-a86f-442c-9162-e36484a5b4cf None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:01.204810 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9bf8c2-65fe-43fd-be11-1df9c411df1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:01.205437 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bf8c2-65fe-43fd-be11-1df9c411df1a None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:01.205718 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bf8c2-65fe-43fd-be11-1df9c411df1a None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:01.205989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bf8c2-65fe-43fd-be11-1df9c411df1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:01.212643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9bf8c2-65fe-43fd-be11-1df9c411df1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:01.214091 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9bf8c2-65fe-43fd-be11-1df9c411df1a None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:01.225627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-cc9bf8c2-65fe-43fd-be11-1df9c411df1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:55:01.225627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:01.225627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:01.225627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:01.225627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:01.225627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:01.225627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:01.225627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:01.225627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:01.225627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:01.225627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:01.225627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:01.225627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:55:01.225627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:55:01.225627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:01.226557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:01.227821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:55:01.227821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:55:01.227821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:55:01.227821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:55:01.227821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:55:01.227821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:55:01.227821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:55:01.227821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:01.231021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 483/987] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:55:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:01.232789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f4a62-0bc6-4bff-93c0-6ecf7229ad55 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:01.339229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f4a62-0bc6-4bff-93c0-6ecf7229ad55 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:01.374507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc05d9-1ffd-47f7-81d3-c521775dbf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:01.375158 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc05d9-1ffd-47f7-81d3-c521775dbf6d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:01.375431 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc05d9-1ffd-47f7-81d3-c521775dbf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:01.375833 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc05d9-1ffd-47f7-81d3-c521775dbf6d None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:01.376386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc05d9-1ffd-47f7-81d3-c521775dbf6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:01.376741 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc05d9-1ffd-47f7-81d3-c521775dbf6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:01.378144 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc05d9-1ffd-47f7-81d3-c521775dbf6d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:01.393835 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 505/988] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:55:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:01.400465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9bf8c2-65fe-43fd-be11-1df9c411df1a None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:01.505949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9bf8c2-65fe-43fd-be11-1df9c411df1a None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:01.538997 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a949c-4c82-4556-a728-7a9391dbc809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:01.539597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a949c-4c82-4556-a728-7a9391dbc809 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:01.539876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a949c-4c82-4556-a728-7a9391dbc809 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:01.540149 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a949c-4c82-4556-a728-7a9391dbc809 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:01.540872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a949c-4c82-4556-a728-7a9391dbc809 None admin] RBAC: Adding query filter params (role.id=c02e5654263d45f88dd97c056d1f9769, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=0bd9c73ff2d94089a34de12c1155e1b5) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:01.541182 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a949c-4c82-4556-a728-7a9391dbc809 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:01.542687 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a949c-4c82-4556-a728-7a9391dbc809 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:01.555010 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 484/989] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:55:01 2022] GET /identity/v3/role_assignments?user.id=0bd9c73ff2d94089a34de12c1155e1b5&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=c02e5654263d45f88dd97c056d1f9769 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:02.423293 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b7eda-b5d3-49b8-a8af-5d881ce55806 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:02.423789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b7eda-b5d3-49b8-a8af-5d881ce55806 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:02.423789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b7eda-b5d3-49b8-a8af-5d881ce55806 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:02.425276 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 506/990] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:02.431497 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d85c96-ed59-4e1e-8848-0cde067b5b7b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:02.431904 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d85c96-ed59-4e1e-8848-0cde067b5b7b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:02.432093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d85c96-ed59-4e1e-8848-0cde067b5b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:02.445944 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:02.452325 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-e2d85c96-ed59-4e1e-8848-0cde067b5b7b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:02.453550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e2d85c96-ed59-4e1e-8848-0cde067b5b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:02.524387 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 485/991] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:02.802423 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b7eda-b5d3-49b8-a8af-5d881ce55806 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:02.860728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a022e8-4c1c-4ab0-a81d-e106ba6003ef None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:02.861043 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a022e8-4c1c-4ab0-a81d-e106ba6003ef None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:02.861278 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a022e8-4c1c-4ab0-a81d-e106ba6003ef None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:02.862045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:02.862631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 486/992] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:02.868329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e20c5-9b9d-4a43-8104-f355e183d818 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:02.868633 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e20c5-9b9d-4a43-8104-f355e183d818 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:02.868900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e20c5-9b9d-4a43-8104-f355e183d818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:02.887631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-da6e20c5-9b9d-4a43-8104-f355e183d818 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:02.888910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-da6e20c5-9b9d-4a43-8104-f355e183d818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:02.917475 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b7eda-b5d3-49b8-a8af-5d881ce55806 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:02.952361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00af48-22ac-4ed8-bed7-90d082f43c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:02.952990 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00af48-22ac-4ed8-bed7-90d082f43c28 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:02.953264 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00af48-22ac-4ed8-bed7-90d082f43c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:02.953533 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00af48-22ac-4ed8-bed7-90d082f43c28 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:02.954105 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00af48-22ac-4ed8-bed7-90d082f43c28 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:02.955489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00af48-22ac-4ed8-bed7-90d082f43c28 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:02.962178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 487/993] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0b00af48-22ac-4ed8-bed7-90d082f43c28 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:02.964816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:02.965781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:02.965781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:02.965781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:02.965781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:02.965781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:02.965781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:02.965781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:55:02.965781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:55:02.965781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:55:02.965781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:55:02.965781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:02.965781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:02.965781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:02.965781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:02.965781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:02.965781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:02.966641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:02.966641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:02.966641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:02.966641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:02.966641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:02.966641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:02.966641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:02.966641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:02.966641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:02.966641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:02.966641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:55:02.966641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:55:02.966641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 07:55:02.966641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:02.966641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 507/994] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:55:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:02.971600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e20c5-9b9d-4a43-8104-f355e183d818 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:03.042157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:03.077449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e20c5-9b9d-4a43-8104-f355e183d818 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:03.113797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a151b-993f-4cde-90db-a35a72067854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:03.114489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a151b-993f-4cde-90db-a35a72067854 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:03.114764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a151b-993f-4cde-90db-a35a72067854 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:03.115061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a151b-993f-4cde-90db-a35a72067854 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:03.115682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a151b-993f-4cde-90db-a35a72067854 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:03.115976 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a151b-993f-4cde-90db-a35a72067854 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:03.117403 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a151b-993f-4cde-90db-a35a72067854 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:03.127041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 488/995] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Oct 6 07:55:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:03.128020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00af48-22ac-4ed8-bed7-90d082f43c28 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:03.233044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00af48-22ac-4ed8-bed7-90d082f43c28 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:03.234124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a151b-993f-4cde-90db-a35a72067854 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:03.252526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:03.272893 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e32dd-344d-45f2-98a0-b948ebaf489f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:03.277589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e32dd-344d-45f2-98a0-b948ebaf489f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:03.277589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e32dd-344d-45f2-98a0-b948ebaf489f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:03.277589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e32dd-344d-45f2-98a0-b948ebaf489f None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:03.277589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e32dd-344d-45f2-98a0-b948ebaf489f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:03.277589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e32dd-344d-45f2-98a0-b948ebaf489f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:03.277589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e32dd-344d-45f2-98a0-b948ebaf489f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:03.282737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 508/996] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Oct 6 07:55:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:03.288940 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e32dd-344d-45f2-98a0-b948ebaf489f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:03.341930 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a151b-993f-4cde-90db-a35a72067854 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:03.372732 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:03.379363 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0da75-adc9-45a8-afdb-df585545f890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:03.380182 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0da75-adc9-45a8-afdb-df585545f890 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:03.380543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0da75-adc9-45a8-afdb-df585545f890 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:03.380822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0da75-adc9-45a8-afdb-df585545f890 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:03.381408 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0da75-adc9-45a8-afdb-df585545f890 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:03.382792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0da75-adc9-45a8-afdb-df585545f890 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-ebf0da75-adc9-45a8-afdb-df585545f890 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:03.392824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:03.394082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:03.394082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:03.394082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:03.394082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:03.394082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:03.394082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:03.394082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:55:03.394082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:55:03.394082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:55:03.394082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:55:03.394082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:03.394082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:03.394082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:03.394082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:03.394082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:03.394082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:03.395503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:03.395503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:03.395503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:03.395503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:03.395503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:03.395503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:03.395503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:03.395503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:03.395503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:03.395503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:03.395503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:55:03.395503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:55:03.395503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 06 07:55:03.395503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:03.395503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 489/997] 10.222.0.30 () {56 vars in 1126 bytes} [Thu Oct 6 07:55:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:03.401043 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0da75-adc9-45a8-afdb-df585545f890 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:03.403467 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e32dd-344d-45f2-98a0-b948ebaf489f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:03.439374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478d2d7-bb52-454b-bce8-6923634f4531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:03.440006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478d2d7-bb52-454b-bce8-6923634f4531 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:03.440259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478d2d7-bb52-454b-bce8-6923634f4531 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:03.440531 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478d2d7-bb52-454b-bce8-6923634f4531 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:03.441220 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478d2d7-bb52-454b-bce8-6923634f4531 None admin] RBAC: Adding query filter params (interface=public, service_id=c362b060c58447b985f4f904c100f7cf, region_id=RegionOne) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:03.441559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478d2d7-bb52-454b-bce8-6923634f4531 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:03.443082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478d2d7-bb52-454b-bce8-6923634f4531 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:03.448253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 509/998] 10.222.0.30 () {56 vars in 1251 bytes} [Thu Oct 6 07:55:03 2022] GET /identity/v3/endpoints?service_id=c362b060c58447b985f4f904c100f7cf&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:03.453810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478d2d7-bb52-454b-bce8-6923634f4531 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:03.502004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:03.520159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0da75-adc9-45a8-afdb-df585545f890 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:03.555820 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb6bb9-eeeb-49bc-8a7f-ff1971764395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:03.556435 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb6bb9-eeeb-49bc-8a7f-ff1971764395 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:03.556716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb6bb9-eeeb-49bc-8a7f-ff1971764395 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:03.556994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb6bb9-eeeb-49bc-8a7f-ff1971764395 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:03.557622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb6bb9-eeeb-49bc-8a7f-ff1971764395 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:03.557922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb6bb9-eeeb-49bc-8a7f-ff1971764395 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:03.559329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb6bb9-eeeb-49bc-8a7f-ff1971764395 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:03.560973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478d2d7-bb52-454b-bce8-6923634f4531 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:03.565208 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 490/999] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Oct 6 07:55:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:03.571960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb6bb9-eeeb-49bc-8a7f-ff1971764395 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:03.598036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9c6c5-d489-46b7-9ab8-e24f929766df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:03.598695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9c6c5-d489-46b7-9ab8-e24f929766df None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:03.598842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9c6c5-d489-46b7-9ab8-e24f929766df None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:03.599119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9c6c5-d489-46b7-9ab8-e24f929766df None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:03.599718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9c6c5-d489-46b7-9ab8-e24f929766df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:03.601100 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9c6c5-d489-46b7-9ab8-e24f929766df None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:03.607681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 510/1000] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Oct 6 07:55:03 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 07:55:03.640075 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:03.693499 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb6bb9-eeeb-49bc-8a7f-ff1971764395 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:03.733889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9533092-13ae-460b-b6a9-9387970351b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:03.734510 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9533092-13ae-460b-b6a9-9387970351b2 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:03.734783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9533092-13ae-460b-b6a9-9387970351b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:03.735081 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9533092-13ae-460b-b6a9-9387970351b2 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:03.735723 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9533092-13ae-460b-b6a9-9387970351b2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:03.736022 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9533092-13ae-460b-b6a9-9387970351b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:03.737442 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9533092-13ae-460b-b6a9-9387970351b2 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:03.744123 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 491/1001] 10.222.0.30 () {56 vars in 1122 bytes} [Thu Oct 6 07:55:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:03.750217 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9c6c5-d489-46b7-9ab8-e24f929766df None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:03.783908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2465a41-5979-4524-9cca-a110b76e47bd None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:03.784191 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2465a41-5979-4524-9cca-a110b76e47bd None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:03.784460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2465a41-5979-4524-9cca-a110b76e47bd None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:03.786855 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 492/1002] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:55:03.791681 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a22584-1e1e-4df6-9865-e23b7bd51067 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:03.791961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a22584-1e1e-4df6-9865-e23b7bd51067 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:03.792258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a22584-1e1e-4df6-9865-e23b7bd51067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:03.811001 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-77a22584-1e1e-4df6-9865-e23b7bd51067 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:03.812868 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-77a22584-1e1e-4df6-9865-e23b7bd51067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:03.872618 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9c6c5-d489-46b7-9ab8-e24f929766df None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:03.883004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:03.891099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 493/1003] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:55:03.907613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2dea4-8edf-4aff-8cb3-a54f1e92e3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:03.908213 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2dea4-8edf-4aff-8cb3-a54f1e92e3b2 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:03.908484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2dea4-8edf-4aff-8cb3-a54f1e92e3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:03.908750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2dea4-8edf-4aff-8cb3-a54f1e92e3b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:03.909418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2dea4-8edf-4aff-8cb3-a54f1e92e3b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:03.910883 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2dea4-8edf-4aff-8cb3-a54f1e92e3b2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:04.022964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 511/1004] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 07:55:03 2022] POST /identity/v3/endpoints => generated 325 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:55:04.170900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a22584-1e1e-4df6-9865-e23b7bd51067 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:04.228440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:04.276814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a22584-1e1e-4df6-9865-e23b7bd51067 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:04.318795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587327e5-5e97-41b9-8eca-208de053cfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:04.319458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587327e5-5e97-41b9-8eca-208de053cfc8 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:04.319734 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587327e5-5e97-41b9-8eca-208de053cfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:04.320007 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587327e5-5e97-41b9-8eca-208de053cfc8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:04.320586 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587327e5-5e97-41b9-8eca-208de053cfc8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:04.321959 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587327e5-5e97-41b9-8eca-208de053cfc8 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-587327e5-5e97-41b9-8eca-208de053cfc8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:04.330913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:04.332573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:04.332573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:04.332573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:04.332573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:04.332573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:04.332573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:04.332573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:55:04.332573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:55:04.332573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:55:04.332573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:55:04.332573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:04.332573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:04.332573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:04.332573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:04.332573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:04.332573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:04.333541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:04.333541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:04.333541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:04.333541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:04.333541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:04.333541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:04.333541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:04.333541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:04.333541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:04.333541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:04.333541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:55:04.333541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:55:04.333541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 07:55:04.333541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:04.333541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 494/1005] 10.222.0.30 () {56 vars in 1105 bytes} [Thu Oct 6 07:55:04 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:04.339310 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2dea4-8edf-4aff-8cb3-a54f1e92e3b2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:04.442346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2dea4-8edf-4aff-8cb3-a54f1e92e3b2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:04.476644 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5fe4d-1a8b-4f4b-90a5-97eb1af99232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:04.477251 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5fe4d-1a8b-4f4b-90a5-97eb1af99232 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:04.477528 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5fe4d-1a8b-4f4b-90a5-97eb1af99232 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:04.477799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5fe4d-1a8b-4f4b-90a5-97eb1af99232 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:04.478492 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5fe4d-1a8b-4f4b-90a5-97eb1af99232 None admin] RBAC: Adding query filter params (name=image) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:04.478799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5fe4d-1a8b-4f4b-90a5-97eb1af99232 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:04.480290 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5fe4d-1a8b-4f4b-90a5-97eb1af99232 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:04.490499 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 512/1006] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:04.491480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587327e5-5e97-41b9-8eca-208de053cfc8 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:04.594278 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587327e5-5e97-41b9-8eca-208de053cfc8 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:04.607726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:04.630713 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609711f0-18c4-408b-b618-7ebf6321fd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:04.631345 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609711f0-18c4-408b-b618-7ebf6321fd76 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:04.631628 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609711f0-18c4-408b-b618-7ebf6321fd76 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:04.631957 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609711f0-18c4-408b-b618-7ebf6321fd76 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:04.632502 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609711f0-18c4-408b-b618-7ebf6321fd76 None admin] RBAC: Adding query filter params (type=image) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:04.632795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609711f0-18c4-408b-b618-7ebf6321fd76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:04.634159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609711f0-18c4-408b-b618-7ebf6321fd76 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:04.643483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 495/1007] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:04 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:05.817538 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cd260-93a5-4cc8-a0b4-8292e8767c10 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:05.818051 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cd260-93a5-4cc8-a0b4-8292e8767c10 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:05.818129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cd260-93a5-4cc8-a0b4-8292e8767c10 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:05.819648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 513/1008] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:05.824819 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d1aac-3416-4ae4-8901-2b0ea59b7997 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:05.825266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d1aac-3416-4ae4-8901-2b0ea59b7997 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:05.825347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d1aac-3416-4ae4-8901-2b0ea59b7997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:05.844165 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-d25d1aac-3416-4ae4-8901-2b0ea59b7997 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:05.845371 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d25d1aac-3416-4ae4-8901-2b0ea59b7997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:05.912640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 496/1009] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:05 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:06.203540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cd260-93a5-4cc8-a0b4-8292e8767c10 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:06.238086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:06.241393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777b0b0-f783-44bc-bec8-7c81509765d4 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:06.241747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777b0b0-f783-44bc-bec8-7c81509765d4 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:06.242124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777b0b0-f783-44bc-bec8-7c81509765d4 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:06.244037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 497/1010] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:55:06.249358 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d1099-4e6e-4e78-ab8e-80f57e351fc2 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:06.249644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d1099-4e6e-4e78-ab8e-80f57e351fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:06.249916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d1099-4e6e-4e78-ab8e-80f57e351fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:06.274895 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-156d1099-4e6e-4e78-ab8e-80f57e351fc2 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:06.276730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-156d1099-4e6e-4e78-ab8e-80f57e351fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:06.329235 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cd260-93a5-4cc8-a0b4-8292e8767c10 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:06.360050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 498/1011] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:06 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:06.375304 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a638d05-8c7c-4ec7-b19d-86d4f033b0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:06.376257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a638d05-8c7c-4ec7-b19d-86d4f033b0cd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:06.376764 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a638d05-8c7c-4ec7-b19d-86d4f033b0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:06.377238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a638d05-8c7c-4ec7-b19d-86d4f033b0cd None admin] PATH_INFO: `/v3/services/compute` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:06.378030 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a638d05-8c7c-4ec7-b19d-86d4f033b0cd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:06.379986 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a638d05-8c7c-4ec7-b19d-86d4f033b0cd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6a638d05-8c7c-4ec7-b19d-86d4f033b0cd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:06.391017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:06.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:06.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:06.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:06.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:06.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:06.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:06.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:55:06.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:55:06.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:55:06.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:55:06.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:06.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:06.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:06.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:06.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:06.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:06.393079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:06.393079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:06.393079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:06.393079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:06.393079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:06.393079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:06.393079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:06.393079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:06.393079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:06.393079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:06.393079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:55:06.393079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:55:06.393079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 06 07:55:06.393079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:06.400980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 514/1012] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:55:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:06.402313 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d1099-4e6e-4e78-ab8e-80f57e351fc2 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:06.439229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:06.515395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d1099-4e6e-4e78-ab8e-80f57e351fc2 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:06.550883 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513cb8d-70ab-4f3f-a5b7-34c0706c1f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:06.551519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513cb8d-70ab-4f3f-a5b7-34c0706c1f0a None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:06.551795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513cb8d-70ab-4f3f-a5b7-34c0706c1f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:06.552067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513cb8d-70ab-4f3f-a5b7-34c0706c1f0a None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:06.552694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513cb8d-70ab-4f3f-a5b7-34c0706c1f0a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:06.552994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513cb8d-70ab-4f3f-a5b7-34c0706c1f0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:06.554454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513cb8d-70ab-4f3f-a5b7-34c0706c1f0a None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:06.559576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 499/1013] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Oct 6 07:55:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:06.565206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a638d05-8c7c-4ec7-b19d-86d4f033b0cd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:06.642889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513cb8d-70ab-4f3f-a5b7-34c0706c1f0a None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:06.665288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:06.671640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a638d05-8c7c-4ec7-b19d-86d4f033b0cd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:06.707789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28345c-9c9f-45e7-b219-29f31eef2ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:06.708489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28345c-9c9f-45e7-b219-29f31eef2ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:06.708762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28345c-9c9f-45e7-b219-29f31eef2ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:06.709055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28345c-9c9f-45e7-b219-29f31eef2ec2 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:06.709908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad28345c-9c9f-45e7-b219-29f31eef2ec2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:06.710030 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad28345c-9c9f-45e7-b219-29f31eef2ec2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:06.711331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad28345c-9c9f-45e7-b219-29f31eef2ec2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:06.722010 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 515/1014] 10.222.0.30 () {56 vars in 1112 bytes} [Thu Oct 6 07:55:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:06.724004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28345c-9c9f-45e7-b219-29f31eef2ec2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:06.757311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:06.760318 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513cb8d-70ab-4f3f-a5b7-34c0706c1f0a None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:06.799993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b570e4-7b21-4245-bacf-c2367fbcd674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:06.800656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b570e4-7b21-4245-bacf-c2367fbcd674 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:06.800941 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b570e4-7b21-4245-bacf-c2367fbcd674 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:06.801208 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b570e4-7b21-4245-bacf-c2367fbcd674 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:06.802333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b570e4-7b21-4245-bacf-c2367fbcd674 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:06.803801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b570e4-7b21-4245-bacf-c2367fbcd674 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:06.805389 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 500/1015] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:06.810917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b570e4-7b21-4245-bacf-c2367fbcd674 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:06.839692 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28345c-9c9f-45e7-b219-29f31eef2ec2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:06.876481 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c981ba0-17c8-408b-80ff-4f84a4bbd46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:06.877072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c981ba0-17c8-408b-80ff-4f84a4bbd46f None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:06.877350 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c981ba0-17c8-408b-80ff-4f84a4bbd46f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:06.877613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c981ba0-17c8-408b-80ff-4f84a4bbd46f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:06.878280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c981ba0-17c8-408b-80ff-4f84a4bbd46f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:06.880417 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c981ba0-17c8-408b-80ff-4f84a4bbd46f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:06.921157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b570e4-7b21-4245-bacf-c2367fbcd674 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:06.952739 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 516/1016] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 07:55:06 2022] POST /identity/v3/endpoints => generated 334 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:55:06.953827 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f5c46-9dc9-47f1-bce1-99c7cedd97e6 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:06.954114 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f5c46-9dc9-47f1-bce1-99c7cedd97e6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:06.954385 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f5c46-9dc9-47f1-bce1-99c7cedd97e6 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:06.955645 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 517/1017] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:06.961605 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7494bb-97db-4955-a8c7-4dd07fefffea None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:06.961841 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7494bb-97db-4955-a8c7-4dd07fefffea None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:06.962110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7494bb-97db-4955-a8c7-4dd07fefffea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:06.972016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba788fb-0b73-4a66-8057-414bf3a50ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:06.973181 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba788fb-0b73-4a66-8057-414bf3a50ddd None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:06.973435 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba788fb-0b73-4a66-8057-414bf3a50ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:06.973716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba788fb-0b73-4a66-8057-414bf3a50ddd None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:06.974443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba788fb-0b73-4a66-8057-414bf3a50ddd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:06.975961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba788fb-0b73-4a66-8057-414bf3a50ddd None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:06.982794 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dd7494bb-97db-4955-a8c7-4dd07fefffea None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:06.984067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dd7494bb-97db-4955-a8c7-4dd07fefffea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:06.997827 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:07.059302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 518/1018] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:06 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:55:07.075309 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 501/1019] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Oct 6 07:55:06 2022] POST /identity/v3/projects => generated 311 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 07:55:07.338614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7494bb-97db-4955-a8c7-4dd07fefffea None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:07.443400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7494bb-97db-4955-a8c7-4dd07fefffea None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:07.479224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6d1b9-78e9-4dc8-90f8-f18fceea6672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:07.479832 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6d1b9-78e9-4dc8-90f8-f18fceea6672 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:07.480106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6d1b9-78e9-4dc8-90f8-f18fceea6672 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:07.480372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6d1b9-78e9-4dc8-90f8-f18fceea6672 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:07.481034 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6d1b9-78e9-4dc8-90f8-f18fceea6672 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:07.482403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6d1b9-78e9-4dc8-90f8-f18fceea6672 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:07.561171 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 519/1020] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Oct 6 07:55:07 2022] POST /identity/v3/services => generated 242 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:55:09.236479 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670eb0e-755d-4cbf-8b33-f735a3c6c779 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:09.237009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670eb0e-755d-4cbf-8b33-f735a3c6c779 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:09.237009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670eb0e-755d-4cbf-8b33-f735a3c6c779 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:09.238430 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 502/1021] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:09.243793 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111768fe-f333-42a7-943f-ab152e2e803f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:09.244193 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111768fe-f333-42a7-943f-ab152e2e803f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:09.244322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111768fe-f333-42a7-943f-ab152e2e803f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:09.258264 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:09.264583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-111768fe-f333-42a7-943f-ab152e2e803f None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:09.266577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-111768fe-f333-42a7-943f-ab152e2e803f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:09.280124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d9f2d-5d8b-4962-93b8-9c6bbe589418 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:09.280409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d9f2d-5d8b-4962-93b8-9c6bbe589418 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:09.280683 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d9f2d-5d8b-4962-93b8-9c6bbe589418 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:09.282013 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 503/1022] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:55:09.287207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dcdd8-030c-42aa-9540-909298e48a35 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:09.287479 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dcdd8-030c-42aa-9540-909298e48a35 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:09.287963 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dcdd8-030c-42aa-9540-909298e48a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:09.306340 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-5f6dcdd8-030c-42aa-9540-909298e48a35 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:09.307642 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5f6dcdd8-030c-42aa-9540-909298e48a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:09.319774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:09.347338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 520/1023] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:09.382391 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 504/1024] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:09.625256 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111768fe-f333-42a7-943f-ab152e2e803f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:09.670104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6dcdd8-030c-42aa-9540-909298e48a35 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:09.745788 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111768fe-f333-42a7-943f-ab152e2e803f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:09.777787 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:09.783002 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbcec5-061f-4404-becd-1292842935ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:09.783641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbcec5-061f-4404-becd-1292842935ff None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:09.783918 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbcec5-061f-4404-becd-1292842935ff None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:09.784266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbcec5-061f-4404-becd-1292842935ff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:09.792166 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6dcdd8-030c-42aa-9540-909298e48a35 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:09.808399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdbcec5-061f-4404-becd-1292842935ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:55:09.808399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-fbdbcec5-061f-4404-becd-1292842935ff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 07:55:09.808399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:09.808399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:09.808399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:09.808399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:09.808399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:09.808399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:09.808399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:09.808399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:09.808399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:09.808399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:09.808399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:09.808399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:09.809252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:09.810185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:09.810185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:09.810185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:09.810185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:09.810185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:09.810185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:09.810185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:55:09.810185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:55:09.810185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:55:09.810185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:55:09.810185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:55:09.810185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:55:09.810185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 07:55:09.810185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:09.810185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 521/1025] 10.222.0.30 () {56 vars in 1120 bytes} [Thu Oct 6 07:55:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:09.811060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007f1c5-9d3e-4b9c-8e19-a116919bd1fe None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:09.811060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007f1c5-9d3e-4b9c-8e19-a116919bd1fe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:09.811060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007f1c5-9d3e-4b9c-8e19-a116919bd1fe None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:09.811060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 522/1026] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:09.811060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007f1c5-9d3e-4b9c-8e19-a116919bd1fe None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:09.832438 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae02435-e2ae-49c0-a58f-277ecf780248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:09.833062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae02435-e2ae-49c0-a58f-277ecf780248 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:09.833336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae02435-e2ae-49c0-a58f-277ecf780248 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:09.833611 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae02435-e2ae-49c0-a58f-277ecf780248 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:09.834711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae02435-e2ae-49c0-a58f-277ecf780248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:09.836182 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae02435-e2ae-49c0-a58f-277ecf780248 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:09.837953 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 505/1027] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:09.838836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3fd57-78b9-44cd-aad1-18e9cc6a74c7 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:09.839135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3fd57-78b9-44cd-aad1-18e9cc6a74c7 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:09.839407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3fd57-78b9-44cd-aad1-18e9cc6a74c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:09.856531 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-0ef3fd57-78b9-44cd-aad1-18e9cc6a74c7 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:09.857712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0ef3fd57-78b9-44cd-aad1-18e9cc6a74c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:09.867540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:09.910616 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007f1c5-9d3e-4b9c-8e19-a116919bd1fe None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:09.924757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 506/1028] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:09.926131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3fd57-78b9-44cd-aad1-18e9cc6a74c7 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:09.947427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be9514-ec9a-4b16-b045-a00a40802e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:09.948024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be9514-ec9a-4b16-b045-a00a40802e6f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:09.948296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be9514-ec9a-4b16-b045-a00a40802e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:09.948559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be9514-ec9a-4b16-b045-a00a40802e6f None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:09.949214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be9514-ec9a-4b16-b045-a00a40802e6f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:09.949508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be9514-ec9a-4b16-b045-a00a40802e6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:09.951196 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be9514-ec9a-4b16-b045-a00a40802e6f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:09.965173 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 523/1029] 10.222.0.30 () {56 vars in 1115 bytes} [Thu Oct 6 07:55:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:09.966158 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be9514-ec9a-4b16-b045-a00a40802e6f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:10.029772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:10.031875 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3fd57-78b9-44cd-aad1-18e9cc6a74c7 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:10.068874 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1fafb-7eea-4a38-872a-733825fe977d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:10.069482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1fafb-7eea-4a38-872a-733825fe977d None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:10.069756 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1fafb-7eea-4a38-872a-733825fe977d None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:10.070025 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1fafb-7eea-4a38-872a-733825fe977d None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:10.070676 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1fafb-7eea-4a38-872a-733825fe977d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:10.072220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1fafb-7eea-4a38-872a-733825fe977d None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:10.092429 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be9514-ec9a-4b16-b045-a00a40802e6f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:10.126420 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09e612-047e-4267-a1f9-a282bf986292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:10.127072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09e612-047e-4267-a1f9-a282bf986292 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:10.127344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09e612-047e-4267-a1f9-a282bf986292 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:10.127606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09e612-047e-4267-a1f9-a282bf986292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:10.138454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09e612-047e-4267-a1f9-a282bf986292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:10.140113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09e612-047e-4267-a1f9-a282bf986292 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:10.157759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-bb09e612-047e-4267-a1f9-a282bf986292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:55:10.157759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:10.157759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:10.157759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:10.157759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:10.157759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:10.157759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:10.157759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:10.157759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:10.157759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:10.157759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:10.157759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:10.157759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:55:10.157759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:55:10.157759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:55:10.157759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:10.164097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:55:10.165261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:55:10.165261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:55:10.165261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:10.165261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 524/1030] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:10.165261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09e612-047e-4267-a1f9-a282bf986292 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:10.178035 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:10.191435 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 507/1031] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:55:09 2022] POST /identity/v3/users => generated 285 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:55:10.198566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1fafb-7eea-4a38-872a-733825fe977d None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:10.279837 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09e612-047e-4267-a1f9-a282bf986292 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:10.290992 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:10.322263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d8611-81fc-40c9-8273-29e1f096e3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:10.323168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d8611-81fc-40c9-8273-29e1f096e3ea None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:10.323591 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d8611-81fc-40c9-8273-29e1f096e3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:10.324005 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d8611-81fc-40c9-8273-29e1f096e3ea None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:10.324630 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1fafb-7eea-4a38-872a-733825fe977d None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:10.325326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d8611-81fc-40c9-8273-29e1f096e3ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:10.325774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d8611-81fc-40c9-8273-29e1f096e3ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:10.327484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d8611-81fc-40c9-8273-29e1f096e3ea None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:10.340478 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 525/1032] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:10.346356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d8611-81fc-40c9-8273-29e1f096e3ea None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:10.370078 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0971a-7002-4433-98fe-a7d6f4250389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:10.370714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0971a-7002-4433-98fe-a7d6f4250389 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:10.371095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0971a-7002-4433-98fe-a7d6f4250389 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:10.371366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0971a-7002-4433-98fe-a7d6f4250389 None admin] PATH_INFO: `/v3/services/image` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:10.371951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0971a-7002-4433-98fe-a7d6f4250389 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:10.374327 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0971a-7002-4433-98fe-a7d6f4250389 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-56a0971a-7002-4433-98fe-a7d6f4250389 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:10.385291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:10.386492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:10.386492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:10.386492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:10.386492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:10.386492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:10.386492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:10.386492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:55:10.386492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:55:10.386492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:55:10.386492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:55:10.386492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:10.386492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:10.386492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:10.386492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:10.386492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:10.386492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:10.387527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:10.387527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:10.387527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:10.387527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:10.387527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:10.387527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:10.387527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:10.387527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:10.387527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:10.387527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:10.387527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:55:10.387527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:55:10.387527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 07:55:10.387527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:10.387527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 508/1033] 10.222.0.30 () {56 vars in 1105 bytes} [Thu Oct 6 07:55:10 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:55:10.393648 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0971a-7002-4433-98fe-a7d6f4250389 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:10.466368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:10.473095 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d8611-81fc-40c9-8273-29e1f096e3ea None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:10.506093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0971a-7002-4433-98fe-a7d6f4250389 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:10.512309 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91e649-18e8-487e-acbd-9d5ab2bc945e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:10.513343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91e649-18e8-487e-acbd-9d5ab2bc945e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:10.513629 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91e649-18e8-487e-acbd-9d5ab2bc945e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:10.513897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91e649-18e8-487e-acbd-9d5ab2bc945e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:10.524320 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac91e649-18e8-487e-acbd-9d5ab2bc945e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:10.525757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac91e649-18e8-487e-acbd-9d5ab2bc945e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:10.538740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ac91e649-18e8-487e-acbd-9d5ab2bc945e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:55:10.538740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:10.538740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:10.538740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:10.538740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:10.538740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:10.538740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:10.538740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:10.538740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:10.538740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:10.538740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:10.538740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:10.538740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:55:10.538740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:55:10.538740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:55:10.538740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:10.539946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:10.541140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:10.541140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:10.541140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:55:10.541140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:55:10.541140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:55:10.541140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:55:10.541140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:55:10.541140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:55:10.541140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:55:10.541140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:10.541140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 526/1034] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Oct 6 07:55:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:55:10.544515 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96ccb8-6f78-42f9-a086-5e66a7b57bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:10.545325 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96ccb8-6f78-42f9-a086-5e66a7b57bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:10.545454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91e649-18e8-487e-acbd-9d5ab2bc945e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:10.545632 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96ccb8-6f78-42f9-a086-5e66a7b57bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:10.545878 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96ccb8-6f78-42f9-a086-5e66a7b57bd0 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:10.546510 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96ccb8-6f78-42f9-a086-5e66a7b57bd0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:10.546817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96ccb8-6f78-42f9-a086-5e66a7b57bd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:10.548246 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96ccb8-6f78-42f9-a086-5e66a7b57bd0 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:10.558225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 509/1035] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:10.559284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96ccb8-6f78-42f9-a086-5e66a7b57bd0 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:10.578246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:10.661864 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91e649-18e8-487e-acbd-9d5ab2bc945e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:10.668581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96ccb8-6f78-42f9-a086-5e66a7b57bd0 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:10.696218 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a1283-a3cf-4a39-bfcc-a864067e039b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:10.696818 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a1283-a3cf-4a39-bfcc-a864067e039b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:10.697085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a1283-a3cf-4a39-bfcc-a864067e039b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:10.697348 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a1283-a3cf-4a39-bfcc-a864067e039b None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:10.697927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:10.698083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a1283-a3cf-4a39-bfcc-a864067e039b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:10.698348 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a1283-a3cf-4a39-bfcc-a864067e039b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:10.699803 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a1283-a3cf-4a39-bfcc-a864067e039b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:10.706564 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a619c-93cd-4f13-930f-d7d20ac4c088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:10.707573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a619c-93cd-4f13-930f-d7d20ac4c088 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:10.708080 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a619c-93cd-4f13-930f-d7d20ac4c088 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:10.708649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a619c-93cd-4f13-930f-d7d20ac4c088 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:10.709276 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a619c-93cd-4f13-930f-d7d20ac4c088 None admin] RBAC: Adding query filter params (type=image) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:10.715075 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a619c-93cd-4f13-930f-d7d20ac4c088 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:10.715075 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a619c-93cd-4f13-930f-d7d20ac4c088 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:10.722123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 527/1036] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Oct 6 07:55:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:10.722969 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 510/1037] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:10.723065 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a1283-a3cf-4a39-bfcc-a864067e039b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:10.723572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a619c-93cd-4f13-930f-d7d20ac4c088 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:10.814646 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:10.826018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a619c-93cd-4f13-930f-d7d20ac4c088 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:10.831242 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a1283-a3cf-4a39-bfcc-a864067e039b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:10.860061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc62b5e-45dd-46b8-be81-3ce057f626fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:10.860656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc62b5e-45dd-46b8-be81-3ce057f626fe None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:10.860942 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc62b5e-45dd-46b8-be81-3ce057f626fe None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:10.861195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc62b5e-45dd-46b8-be81-3ce057f626fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:10.870504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c03a2-3261-4cff-9986-989fcec71bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:10.871500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c03a2-3261-4cff-9986-989fcec71bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:10.871885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c03a2-3261-4cff-9986-989fcec71bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:10.872110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c03a2-3261-4cff-9986-989fcec71bf8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:10.872811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c03a2-3261-4cff-9986-989fcec71bf8 None admin] RBAC: Adding query filter params (interface=public, service_id=c991926fb0ab4a92a8e8fabd2f1a793a, region_id=RegionOne) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:10.873377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c03a2-3261-4cff-9986-989fcec71bf8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:10.874840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c03a2-3261-4cff-9986-989fcec71bf8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:10.881753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 528/1038] 10.222.0.30 () {56 vars in 1251 bytes} [Thu Oct 6 07:55:10 2022] GET /identity/v3/endpoints?service_id=c991926fb0ab4a92a8e8fabd2f1a793a&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:10.887337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c03a2-3261-4cff-9986-989fcec71bf8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:10.932127 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:10.941469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc62b5e-45dd-46b8-be81-3ce057f626fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:10.943131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc62b5e-45dd-46b8-be81-3ce057f626fe None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:11.001516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c03a2-3261-4cff-9986-989fcec71bf8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:11.038023 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd9811-8427-4573-9ef2-09e96c8cda11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:11.038626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd9811-8427-4573-9ef2-09e96c8cda11 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:11.038898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd9811-8427-4573-9ef2-09e96c8cda11 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:11.039367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd9811-8427-4573-9ef2-09e96c8cda11 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:11.039954 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd9811-8427-4573-9ef2-09e96c8cda11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:11.041305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd9811-8427-4573-9ef2-09e96c8cda11 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:11.044280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 511/1039] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:55:10 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 06 07:55:11.049690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc62b5e-45dd-46b8-be81-3ce057f626fe None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:11.053308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 529/1040] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Oct 6 07:55:10 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 07:55:11.179463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:11.188722 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc62b5e-45dd-46b8-be81-3ce057f626fe None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:11.226325 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61820feb-2a14-4a58-b3fe-ca5e7d59efa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:11.226973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61820feb-2a14-4a58-b3fe-ca5e7d59efa0 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:11.227252 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61820feb-2a14-4a58-b3fe-ca5e7d59efa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:11.227520 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61820feb-2a14-4a58-b3fe-ca5e7d59efa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:11.239440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61820feb-2a14-4a58-b3fe-ca5e7d59efa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:11.240890 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61820feb-2a14-4a58-b3fe-ca5e7d59efa0 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:11.256338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-61820feb-2a14-4a58-b3fe-ca5e7d59efa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:55:11.256338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:11.256338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:11.256338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:11.256338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:11.256338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:11.256338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:11.256338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:11.256338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:11.256338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:11.256338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:11.256338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:11.256338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:55:11.256338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:55:11.256338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:55:11.256338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:11.257329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:55:11.258239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:55:11.258239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:55:11.258239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:11.258239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 512/1041] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:55:11.263292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd9811-8427-4573-9ef2-09e96c8cda11 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:11.279418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:11.365080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd9811-8427-4573-9ef2-09e96c8cda11 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:11.399848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76152389-03d7-44cd-97ca-06ef1c7cd876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:11.400449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76152389-03d7-44cd-97ca-06ef1c7cd876 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:11.400718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76152389-03d7-44cd-97ca-06ef1c7cd876 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:11.400979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76152389-03d7-44cd-97ca-06ef1c7cd876 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:11.401623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76152389-03d7-44cd-97ca-06ef1c7cd876 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:11.401916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76152389-03d7-44cd-97ca-06ef1c7cd876 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:11.403307 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76152389-03d7-44cd-97ca-06ef1c7cd876 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:11.415901 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 530/1042] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:11.421271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61820feb-2a14-4a58-b3fe-ca5e7d59efa0 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:11.523604 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61820feb-2a14-4a58-b3fe-ca5e7d59efa0 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:11.558764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3aaae5-0b46-4fd0-9c5b-c4a135e957e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:11.559421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3aaae5-0b46-4fd0-9c5b-c4a135e957e5 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:11.559693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3aaae5-0b46-4fd0-9c5b-c4a135e957e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:11.559977 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3aaae5-0b46-4fd0-9c5b-c4a135e957e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:11.566617 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3aaae5-0b46-4fd0-9c5b-c4a135e957e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:11.568091 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3aaae5-0b46-4fd0-9c5b-c4a135e957e5 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:11.579321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-6e3aaae5-0b46-4fd0-9c5b-c4a135e957e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:55:11.579321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:11.579321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:11.579321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:11.579321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:11.579321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:11.579321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:11.579321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:11.579321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:11.579321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:11.579321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:11.579321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:11.579321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:55:11.579321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:55:11.579321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:11.580255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:11.581267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:55:11.581267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:55:11.581267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:55:11.581267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:55:11.581267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:55:11.581267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:55:11.581267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:55:11.581267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:11.581267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 513/1043] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:55:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:11.586495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76152389-03d7-44cd-97ca-06ef1c7cd876 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:11.660619 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:11.691197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76152389-03d7-44cd-97ca-06ef1c7cd876 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:11.725295 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360b642-28d7-4e3d-8a54-3faef8debaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:11.725894 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360b642-28d7-4e3d-8a54-3faef8debaf3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:11.726170 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360b642-28d7-4e3d-8a54-3faef8debaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:11.726433 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360b642-28d7-4e3d-8a54-3faef8debaf3 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:11.727144 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360b642-28d7-4e3d-8a54-3faef8debaf3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:11.727443 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360b642-28d7-4e3d-8a54-3faef8debaf3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:11.728844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360b642-28d7-4e3d-8a54-3faef8debaf3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:11.741656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 531/1044] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:55:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:11.747838 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3aaae5-0b46-4fd0-9c5b-c4a135e957e5 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:11.751862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:11.848877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3aaae5-0b46-4fd0-9c5b-c4a135e957e5 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:11.881512 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bb291-7dd9-47e1-af65-2c2e14b9ed6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:11.882107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bb291-7dd9-47e1-af65-2c2e14b9ed6c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:11.882381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bb291-7dd9-47e1-af65-2c2e14b9ed6c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:11.882646 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bb291-7dd9-47e1-af65-2c2e14b9ed6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:11.883384 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bb291-7dd9-47e1-af65-2c2e14b9ed6c None admin] RBAC: Adding query filter params (role.id=c02e5654263d45f88dd97c056d1f9769, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=209795285384486183cda455fe316bc9) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:11.883683 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bb291-7dd9-47e1-af65-2c2e14b9ed6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:11.885109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bb291-7dd9-47e1-af65-2c2e14b9ed6c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:11.890372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 514/1045] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:55:11 2022] GET /identity/v3/role_assignments?user.id=209795285384486183cda455fe316bc9&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=c02e5654263d45f88dd97c056d1f9769 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:12.459405 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c22b24-77cc-471c-b966-4a5866b8fa78 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:12.459895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c22b24-77cc-471c-b966-4a5866b8fa78 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:12.459895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c22b24-77cc-471c-b966-4a5866b8fa78 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:12.461217 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 532/1046] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:12.466739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46c83f-fbba-412b-8e2d-66e46f6ba10b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:12.467154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46c83f-fbba-412b-8e2d-66e46f6ba10b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:12.467327 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46c83f-fbba-412b-8e2d-66e46f6ba10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:12.486134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-6c46c83f-fbba-412b-8e2d-66e46f6ba10b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:12.487346 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6c46c83f-fbba-412b-8e2d-66e46f6ba10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:12.549663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:12.556378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 515/1047] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:12.827123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c22b24-77cc-471c-b966-4a5866b8fa78 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:12.936472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c22b24-77cc-471c-b966-4a5866b8fa78 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:12.966845 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:12.971947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c66c86-551e-4650-8be9-d14f7343adc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:12.973159 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c66c86-551e-4650-8be9-d14f7343adc2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:12.973389 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c66c86-551e-4650-8be9-d14f7343adc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:12.973643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c66c86-551e-4650-8be9-d14f7343adc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:12.983599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c66c86-551e-4650-8be9-d14f7343adc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:55:12.988979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e4c66c86-551e-4650-8be9-d14f7343adc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:55:12.988979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:12.988979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:12.988979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:12.988979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:12.988979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:12.988979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:12.988979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:12.988979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:12.988979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:12.988979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:12.988979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:12.988979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:55:12.988979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:55:12.988979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:55:12.988979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:55:12.989922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:55:12.989922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:55:12.989922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:55:12.989922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:55:12.989922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:12.989922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:12.989922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:12.989922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:12.989922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:12.989922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:12.989922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:12.989922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:12.989922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:12.989922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:12.989922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:12.989922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:12.990802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:12.990802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:12.990802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:12.990802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:12.990802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:55:12.990802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:55:12.990802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:55:12.990802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:55:12.990802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:55:12.990802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:55:12.990802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:55:12.990802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:12.992102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 533/1048] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:55:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:12.996216 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46c83f-fbba-412b-8e2d-66e46f6ba10b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:13.103805 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46c83f-fbba-412b-8e2d-66e46f6ba10b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:13.151857 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1871090-2053-4929-9516-3758a80e35b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:13.152941 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1871090-2053-4929-9516-3758a80e35b9 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:13.152941 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1871090-2053-4929-9516-3758a80e35b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:13.153079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1871090-2053-4929-9516-3758a80e35b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:13.153669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1871090-2053-4929-9516-3758a80e35b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:13.154000 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1871090-2053-4929-9516-3758a80e35b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:13.155418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1871090-2053-4929-9516-3758a80e35b9 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:13.169492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 516/1049] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:55:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:13.171110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c66c86-551e-4650-8be9-d14f7343adc2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:13.231476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb6710-b856-42b9-87c5-cbba5df0ade4 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:13.231786 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb6710-b856-42b9-87c5-cbba5df0ade4 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:13.232024 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb6710-b856-42b9-87c5-cbba5df0ade4 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:13.233370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 517/1050] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:55:13.238829 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64325793-7d08-4739-8217-c22f63aef6dd None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:13.239151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64325793-7d08-4739-8217-c22f63aef6dd None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:13.239429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64325793-7d08-4739-8217-c22f63aef6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:13.259741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-64325793-7d08-4739-8217-c22f63aef6dd None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:13.261062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-64325793-7d08-4739-8217-c22f63aef6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:13.278611 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c66c86-551e-4650-8be9-d14f7343adc2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:13.316515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe332e4-2636-485d-ae44-3e38e53826a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:13.317147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe332e4-2636-485d-ae44-3e38e53826a6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:13.317427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe332e4-2636-485d-ae44-3e38e53826a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:13.318220 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe332e4-2636-485d-ae44-3e38e53826a6 None admin] PATH_INFO: `/v3/users/1dffeb63ae9242609ce67404c6776e96` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:13.320558 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:13.332632 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 518/1051] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:55:13.337177 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe332e4-2636-485d-ae44-3e38e53826a6 None admin] RBAC: Authorizing `identity:get_user(user_id=1dffeb63ae9242609ce67404c6776e96)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:13.338677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe332e4-2636-485d-ae44-3e38e53826a6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:13.344941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 534/1052] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:55:13 2022] GET /identity/v3/users/1dffeb63ae9242609ce67404c6776e96 => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:13.345913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64325793-7d08-4739-8217-c22f63aef6dd None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:13.445487 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64325793-7d08-4739-8217-c22f63aef6dd None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:13.481627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f0f61-5999-432e-9c5b-f043a470b231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:13.482260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f0f61-5999-432e-9c5b-f043a470b231 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:13.482492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f0f61-5999-432e-9c5b-f043a470b231 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:13.482761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f0f61-5999-432e-9c5b-f043a470b231 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:13.540355 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:13.550393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f0f61-5999-432e-9c5b-f043a470b231 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:13.554737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f0f61-5999-432e-9c5b-f043a470b231 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:13.628054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe332e4-2636-485d-ae44-3e38e53826a6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:13.655494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 519/1053] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:55:13 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 07:55:13.661071 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f0f61-5999-432e-9c5b-f043a470b231 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:13.688396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:13.750259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe332e4-2636-485d-ae44-3e38e53826a6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:13.767307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:13.780237 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f0f61-5999-432e-9c5b-f043a470b231 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:13.787484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdf19a-6c87-4877-a651-c6d0d92b950e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:13.788123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf19a-6c87-4877-a651-c6d0d92b950e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:13.788406 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf19a-6c87-4877-a651-c6d0d92b950e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:13.788669 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf19a-6c87-4877-a651-c6d0d92b950e None admin] PATH_INFO: `/v3/services/image` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:13.789257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdf19a-6c87-4877-a651-c6d0d92b950e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:13.790640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdf19a-6c87-4877-a651-c6d0d92b950e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5bfdf19a-6c87-4877-a651-c6d0d92b950e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:13.799710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:13.801436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:13.801436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:13.801436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:13.801436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:13.801436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:13.801436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:13.801436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:55:13.801436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:55:13.801436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:55:13.801436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:55:13.801436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:13.801436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:13.801436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:13.801436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:13.801436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:13.801436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:13.802352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:13.802352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:13.802352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:13.802352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:13.802352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:13.802352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:13.802352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:13.802352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:13.802352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:13.802352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:13.802352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:55:13.802352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:55:13.802352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 06 07:55:13.802352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:13.802352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 535/1054] 10.222.0.30 () {56 vars in 1105 bytes} [Thu Oct 6 07:55:13 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:55:13.807754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdf19a-6c87-4877-a651-c6d0d92b950e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:13.821228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2faecc7-45b0-4971-8306-6fa2ab69091f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:13.821864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faecc7-45b0-4971-8306-6fa2ab69091f None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:13.822141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faecc7-45b0-4971-8306-6fa2ab69091f None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:13.822413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faecc7-45b0-4971-8306-6fa2ab69091f None admin] PATH_INFO: `/v3/projects/e77865d721474450a4f95f9bf7e9f0d1` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:13.823541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2faecc7-45b0-4971-8306-6fa2ab69091f None admin] RBAC: Authorizing `identity:get_project(project_id=e77865d721474450a4f95f9bf7e9f0d1)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:13.824988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2faecc7-45b0-4971-8306-6fa2ab69091f None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:13.831533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 520/1055] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:55:13 2022] GET /identity/v3/projects/e77865d721474450a4f95f9bf7e9f0d1 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:13.832546 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2faecc7-45b0-4971-8306-6fa2ab69091f None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:13.927155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdf19a-6c87-4877-a651-c6d0d92b950e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:13.944206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2faecc7-45b0-4971-8306-6fa2ab69091f None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:13.944884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:13.967833 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c09c8-eea3-4a1a-a900-e6e335e05c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:13.968769 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c09c8-eea3-4a1a-a900-e6e335e05c9f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:13.969227 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c09c8-eea3-4a1a-a900-e6e335e05c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:13.969673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c09c8-eea3-4a1a-a900-e6e335e05c9f None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:13.970523 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c09c8-eea3-4a1a-a900-e6e335e05c9f None admin] RBAC: Adding query filter params (name=image) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:13.971036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c09c8-eea3-4a1a-a900-e6e335e05c9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:13.973151 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c09c8-eea3-4a1a-a900-e6e335e05c9f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:13.980349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 536/1056] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:13.984497 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a75af-df3a-4aee-8159-eedc9a5f6836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:13.985133 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a75af-df3a-4aee-8159-eedc9a5f6836 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:13.985390 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a75af-df3a-4aee-8159-eedc9a5f6836 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:13.985661 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a75af-df3a-4aee-8159-eedc9a5f6836 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:13.986351 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c09c8-eea3-4a1a-a900-e6e335e05c9f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:13.986468 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a75af-df3a-4aee-8159-eedc9a5f6836 None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=e77865d721474450a4f95f9bf7e9f0d1, user.id=1dffeb63ae9242609ce67404c6776e96) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:13.986685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a75af-df3a-4aee-8159-eedc9a5f6836 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:13.988848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a75af-df3a-4aee-8159-eedc9a5f6836 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:13.996876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 521/1057] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:55:13 2022] GET /identity/v3/role_assignments?user.id=1dffeb63ae9242609ce67404c6776e96&scope.project.id=e77865d721474450a4f95f9bf7e9f0d1&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:14.072334 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd86433-6add-4e61-a6df-64a651a6bd8b None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:14.072528 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd86433-6add-4e61-a6df-64a651a6bd8b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:14.072770 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd86433-6add-4e61-a6df-64a651a6bd8b None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:14.074427 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 522/1058] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:55:14.080017 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dda889-d115-4a41-a5f9-2aeb5f04bc50 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:14.080324 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dda889-d115-4a41-a5f9-2aeb5f04bc50 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:14.080561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dda889-d115-4a41-a5f9-2aeb5f04bc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:14.108742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c09c8-eea3-4a1a-a900-e6e335e05c9f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:14.125258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-a3dda889-d115-4a41-a5f9-2aeb5f04bc50 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:14.126477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a3dda889-d115-4a41-a5f9-2aeb5f04bc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:14.150619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:14.152157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f47112-63ad-4576-8160-49cac6d80a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:14.152876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f47112-63ad-4576-8160-49cac6d80a21 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:14.153164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f47112-63ad-4576-8160-49cac6d80a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:14.153773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f47112-63ad-4576-8160-49cac6d80a21 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:14.154045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f47112-63ad-4576-8160-49cac6d80a21 None admin] RBAC: Adding query filter params (type=image) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:14.154342 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f47112-63ad-4576-8160-49cac6d80a21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:14.155761 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f47112-63ad-4576-8160-49cac6d80a21 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:14.161857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 537/1059] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:14.167627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f47112-63ad-4576-8160-49cac6d80a21 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:14.213484 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 523/1060] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:14 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:55:14.214392 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:14.272476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f47112-63ad-4576-8160-49cac6d80a21 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:14.307082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67e052-3ea5-43b9-82ee-0027e235eb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:14.307682 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67e052-3ea5-43b9-82ee-0027e235eb51 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:14.308014 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67e052-3ea5-43b9-82ee-0027e235eb51 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:14.308284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67e052-3ea5-43b9-82ee-0027e235eb51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:14.308947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67e052-3ea5-43b9-82ee-0027e235eb51 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:14.310310 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67e052-3ea5-43b9-82ee-0027e235eb51 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:14.395065 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 538/1061] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 07:55:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 07:55:14.515611 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dda889-d115-4a41-a5f9-2aeb5f04bc50 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:14.631701 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dda889-d115-4a41-a5f9-2aeb5f04bc50 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:14.681025 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90c937-2b0b-4253-98cd-41e0ab76e7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:14.681580 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90c937-2b0b-4253-98cd-41e0ab76e7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:14.681850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90c937-2b0b-4253-98cd-41e0ab76e7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:14.682115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90c937-2b0b-4253-98cd-41e0ab76e7a0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:14.695135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90c937-2b0b-4253-98cd-41e0ab76e7a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:55:14.701220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-0d90c937-2b0b-4253-98cd-41e0ab76e7a0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 07:55:14.701220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:14.701220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:14.701220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:14.701220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:14.701220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:14.701220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:14.701220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:14.701220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:14.701220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:14.701220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:14.701220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:14.701220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:55:14.701220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:55:14.701220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:55:14.701220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:55:14.702274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:55:14.702274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:55:14.702274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:55:14.702274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:55:14.702274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:14.702274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:14.702274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:14.702274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:14.702274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:14.702274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:14.702274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:14.702274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:14.702274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:14.702274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:14.702274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:14.702274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:14.703230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:14.703230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:14.703230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:14.703230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:14.703230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:55:14.703230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:55:14.703230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:55:14.703230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:55:14.703230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:55:14.703230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:55:14.703230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 07:55:14.703230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:14.703230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 524/1062] 10.222.0.30 () {56 vars in 1120 bytes} [Thu Oct 6 07:55:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:14.712654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67e052-3ea5-43b9-82ee-0027e235eb51 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:14.845157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:14.851898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67e052-3ea5-43b9-82ee-0027e235eb51 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:14.899387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc72392-8101-43b4-bd39-99fa20381b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:14.900340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc72392-8101-43b4-bd39-99fa20381b73 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:14.900855 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc72392-8101-43b4-bd39-99fa20381b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:14.901283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc72392-8101-43b4-bd39-99fa20381b73 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:14.902105 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc72392-8101-43b4-bd39-99fa20381b73 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:14.902572 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc72392-8101-43b4-bd39-99fa20381b73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:14.904122 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc72392-8101-43b4-bd39-99fa20381b73 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:14.928626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 539/1063] 10.222.0.30 () {56 vars in 1115 bytes} [Thu Oct 6 07:55:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:14.930179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90c937-2b0b-4253-98cd-41e0ab76e7a0 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:14.943508 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:15.054643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90c937-2b0b-4253-98cd-41e0ab76e7a0 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:15.098250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fefa4-5509-4f82-9462-4f9714e02b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:15.098885 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fefa4-5509-4f82-9462-4f9714e02b38 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:15.099194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fefa4-5509-4f82-9462-4f9714e02b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:15.099466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fefa4-5509-4f82-9462-4f9714e02b38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:15.120004 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16fefa4-5509-4f82-9462-4f9714e02b38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:15.121960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16fefa4-5509-4f82-9462-4f9714e02b38 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:15.146974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-d16fefa4-5509-4f82-9462-4f9714e02b38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:55:15.146974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:15.146974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:15.146974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:15.146974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:15.146974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:15.146974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:15.146974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:15.146974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:15.146974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:15.146974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:15.146974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:15.146974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:55:15.146974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:55:15.146974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:55:15.146974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:15.148016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:55:15.148924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:55:15.148924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:55:15.148924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:15.148924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 525/1064] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:15.158009 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc72392-8101-43b4-bd39-99fa20381b73 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:15.286149 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc72392-8101-43b4-bd39-99fa20381b73 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:15.327785 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab6844-ab58-4ad1-ab5a-b32302e51865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:15.328614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab6844-ab58-4ad1-ab5a-b32302e51865 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:15.328892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab6844-ab58-4ad1-ab5a-b32302e51865 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:15.329159 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab6844-ab58-4ad1-ab5a-b32302e51865 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:15.329804 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab6844-ab58-4ad1-ab5a-b32302e51865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:15.330104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab6844-ab58-4ad1-ab5a-b32302e51865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:15.332092 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab6844-ab58-4ad1-ab5a-b32302e51865 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:15.345799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 540/1065] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:15.352202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fefa4-5509-4f82-9462-4f9714e02b38 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:15.425127 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:15.472053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fefa4-5509-4f82-9462-4f9714e02b38 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:15.519391 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1e742-52dc-4171-85ff-8ecead3731fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:15.520097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1e742-52dc-4171-85ff-8ecead3731fa None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:15.520344 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1e742-52dc-4171-85ff-8ecead3731fa None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:15.520591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1e742-52dc-4171-85ff-8ecead3731fa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:15.528464 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1e742-52dc-4171-85ff-8ecead3731fa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:15.529897 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1e742-52dc-4171-85ff-8ecead3731fa None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:15.541416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-b5d1e742-52dc-4171-85ff-8ecead3731fa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:55:15.541416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:15.541416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:15.541416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:15.541416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:15.541416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:15.541416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:15.541416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:15.541416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:15.541416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:15.541416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:15.541416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:15.541416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:55:15.541416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:55:15.541416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:55:15.541416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:15.542347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:15.543361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:15.543361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:15.543361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:55:15.543361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:55:15.543361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:55:15.543361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:55:15.543361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:55:15.543361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:55:15.543361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:55:15.543361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:15.548764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 526/1066] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Oct 6 07:55:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:15.548891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab6844-ab58-4ad1-ab5a-b32302e51865 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:15.572455 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:15.650733 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab6844-ab58-4ad1-ab5a-b32302e51865 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:15.684760 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7ae92-5bd1-41fd-a482-0c01581bf97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:15.685469 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7ae92-5bd1-41fd-a482-0c01581bf97d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:15.685655 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7ae92-5bd1-41fd-a482-0c01581bf97d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:15.685901 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7ae92-5bd1-41fd-a482-0c01581bf97d None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:15.686626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7ae92-5bd1-41fd-a482-0c01581bf97d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:15.686924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7ae92-5bd1-41fd-a482-0c01581bf97d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:15.688374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7ae92-5bd1-41fd-a482-0c01581bf97d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:15.711033 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 541/1067] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Oct 6 07:55:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:15.712778 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1e742-52dc-4171-85ff-8ecead3731fa None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:15.825751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1e742-52dc-4171-85ff-8ecead3731fa None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:15.846753 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:15.861187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79105b9e-e978-4c7b-b7d8-7c16090854f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:15.861825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79105b9e-e978-4c7b-b7d8-7c16090854f6 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:15.862101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79105b9e-e978-4c7b-b7d8-7c16090854f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:15.862370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79105b9e-e978-4c7b-b7d8-7c16090854f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:15.875051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79105b9e-e978-4c7b-b7d8-7c16090854f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:15.876527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79105b9e-e978-4c7b-b7d8-7c16090854f6 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:15.891561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-79105b9e-e978-4c7b-b7d8-7c16090854f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:55:15.891561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:15.891561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:15.891561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:15.891561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:15.891561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:15.891561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:15.891561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:15.891561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:15.891561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:15.891561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:15.891561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:15.891561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:55:15.891561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:55:15.891561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:55:15.891561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:15.892549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:55:15.893466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:55:15.893466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:55:15.893466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:15.898117 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 527/1068] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:15.899500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7ae92-5bd1-41fd-a482-0c01581bf97d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:15.981649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:16.003747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7ae92-5bd1-41fd-a482-0c01581bf97d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:16.038455 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5015fb-fd00-4611-b4dd-14e7b6a155ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:16.039083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5015fb-fd00-4611-b4dd-14e7b6a155ba None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:16.039363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5015fb-fd00-4611-b4dd-14e7b6a155ba None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:16.039633 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5015fb-fd00-4611-b4dd-14e7b6a155ba None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:16.040272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5015fb-fd00-4611-b4dd-14e7b6a155ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:16.040574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5015fb-fd00-4611-b4dd-14e7b6a155ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:16.041946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5015fb-fd00-4611-b4dd-14e7b6a155ba None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:16.058643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 542/1069] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:16.060054 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79105b9e-e978-4c7b-b7d8-7c16090854f6 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:16.169930 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79105b9e-e978-4c7b-b7d8-7c16090854f6 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:16.203373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bc662-cfe3-4d6f-8216-7735131d8e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:16.204022 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bc662-cfe3-4d6f-8216-7735131d8e07 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:16.204295 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bc662-cfe3-4d6f-8216-7735131d8e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:16.204560 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bc662-cfe3-4d6f-8216-7735131d8e07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:16.212362 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31bc662-cfe3-4d6f-8216-7735131d8e07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:16.213868 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31bc662-cfe3-4d6f-8216-7735131d8e07 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:16.227386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-a31bc662-cfe3-4d6f-8216-7735131d8e07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:55:16.227386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:16.227386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:16.227386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:16.227386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:16.227386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:16.227386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:16.227386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:16.227386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:16.227386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:16.227386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:16.227386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:16.227386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:55:16.227386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:55:16.227386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:16.228379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:16.229301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:55:16.229301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:55:16.229301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:55:16.229301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:55:16.229301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:55:16.229301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:55:16.229301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:55:16.229301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:16.231306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 528/1070] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:55:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:16.234469 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5015fb-fd00-4611-b4dd-14e7b6a155ba None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:16.329221 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed07b3-54c8-4885-929a-dbd0c972d664 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:16.329494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed07b3-54c8-4885-929a-dbd0c972d664 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:16.329761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed07b3-54c8-4885-929a-dbd0c972d664 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:16.333357 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 529/1071] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:55:16.337262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026af223-3b7f-40b3-8712-422360dd21e2 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:16.337815 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026af223-3b7f-40b3-8712-422360dd21e2 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:16.338089 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026af223-3b7f-40b3-8712-422360dd21e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:16.342098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5015fb-fd00-4611-b4dd-14e7b6a155ba None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:16.361540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-026af223-3b7f-40b3-8712-422360dd21e2 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:16.363080 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-026af223-3b7f-40b3-8712-422360dd21e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:16.377693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f469d2-e5bb-451c-94da-92ab617eff6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:16.378549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f469d2-e5bb-451c-94da-92ab617eff6c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:16.378684 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f469d2-e5bb-451c-94da-92ab617eff6c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:16.378862 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f469d2-e5bb-451c-94da-92ab617eff6c None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:16.379759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f469d2-e5bb-451c-94da-92ab617eff6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:16.380068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f469d2-e5bb-451c-94da-92ab617eff6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:16.381491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f469d2-e5bb-451c-94da-92ab617eff6c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:16.389270 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:16.395153 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 543/1072] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:55:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:16.401769 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f469d2-e5bb-451c-94da-92ab617eff6c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:16.403630 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:16.446051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 530/1073] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:16.507399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f469d2-e5bb-451c-94da-92ab617eff6c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:16.540808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b1927-653f-4cac-a3fb-30a19d928e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:16.541453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b1927-653f-4cac-a3fb-30a19d928e9b None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:16.541726 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b1927-653f-4cac-a3fb-30a19d928e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:16.541994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b1927-653f-4cac-a3fb-30a19d928e9b None admin] PATH_INFO: `/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/209795285384486183cda455fe316bc9/roles/c02e5654263d45f88dd97c056d1f9769` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:16.544147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b1927-653f-4cac-a3fb-30a19d928e9b None admin] RBAC: Authorizing `identity:create_grant(project_id=399f0095cc8b4c64893ccc75af13c52d, user_id=209795285384486183cda455fe316bc9, role_id=c02e5654263d45f88dd97c056d1f9769)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:16.545565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b1927-653f-4cac-a3fb-30a19d928e9b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:16.621443 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 544/1074] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:55:16 2022] PUT /identity/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/209795285384486183cda455fe316bc9/roles/c02e5654263d45f88dd97c056d1f9769 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 07:55:16.723660 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026af223-3b7f-40b3-8712-422360dd21e2 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:16.759861 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea3a69-0929-4dcb-b59c-4f68b20c1983 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:16.760198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea3a69-0929-4dcb-b59c-4f68b20c1983 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:16.760416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea3a69-0929-4dcb-b59c-4f68b20c1983 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:16.762118 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 545/1075] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:55:16.767835 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6e6ac-eef5-4653-817a-cea27e4ccf7e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:16.768120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6e6ac-eef5-4653-817a-cea27e4ccf7e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:16.768395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6e6ac-eef5-4653-817a-cea27e4ccf7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:16.793109 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3ac6e6ac-eef5-4653-817a-cea27e4ccf7e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:16.793405 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3ac6e6ac-eef5-4653-817a-cea27e4ccf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:16.838769 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026af223-3b7f-40b3-8712-422360dd21e2 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:16.859201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:16.874782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 546/1076] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:55:16.876383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5b5ee-211d-40c0-8eaf-0469e28bf96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:16.877062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5b5ee-211d-40c0-8eaf-0469e28bf96c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:16.877309 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5b5ee-211d-40c0-8eaf-0469e28bf96c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:16.877578 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5b5ee-211d-40c0-8eaf-0469e28bf96c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:16.888448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5b5ee-211d-40c0-8eaf-0469e28bf96c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:55:16.893298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-11c5b5ee-211d-40c0-8eaf-0469e28bf96c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:55:16.893298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:16.893298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:16.893298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:16.893298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:16.893298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:16.893298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:16.893298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:16.893298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:16.893298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:16.893298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:16.893298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:16.893298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:55:16.893298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:55:16.893298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:55:16.893298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:55:16.894280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:55:16.894280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:55:16.894280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:55:16.894280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:55:16.894280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:16.894280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:16.894280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:16.894280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:16.894280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:16.894280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:16.894280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:16.894280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:16.894280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:16.894280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:16.894280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:16.894280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:16.899402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:16.899402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:16.899402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:16.899402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:16.899402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:55:16.899402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:55:16.899402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:55:16.899402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:55:16.899402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:55:16.899402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:55:16.899402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:55:16.899402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:16.899402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 531/1077] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:55:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:16.903337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6e6ac-eef5-4653-817a-cea27e4ccf7e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:17.006629 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6e6ac-eef5-4653-817a-cea27e4ccf7e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:17.040767 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323576d0-9ca8-403a-b707-358911508fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:17.041362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323576d0-9ca8-403a-b707-358911508fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:17.041633 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323576d0-9ca8-403a-b707-358911508fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:17.041896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323576d0-9ca8-403a-b707-358911508fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:17.042549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323576d0-9ca8-403a-b707-358911508fa7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:17.042840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323576d0-9ca8-403a-b707-358911508fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:17.044221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323576d0-9ca8-403a-b707-358911508fa7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:17.052824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 547/1078] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:55:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:17.058407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5b5ee-211d-40c0-8eaf-0469e28bf96c None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:17.083939 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:17.152736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323576d0-9ca8-403a-b707-358911508fa7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:17.175421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5b5ee-211d-40c0-8eaf-0469e28bf96c None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:17.213755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:17.214791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2c05d-0cdf-4d17-9cd4-28d09f0248c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:17.215467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2c05d-0cdf-4d17-9cd4-28d09f0248c7 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:17.215739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2c05d-0cdf-4d17-9cd4-28d09f0248c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:17.216008 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2c05d-0cdf-4d17-9cd4-28d09f0248c7 None admin] PATH_INFO: `/v3/users/1dffeb63ae9242609ce67404c6776e96` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:17.217220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2c05d-0cdf-4d17-9cd4-28d09f0248c7 None admin] RBAC: Authorizing `identity:get_user(user_id=1dffeb63ae9242609ce67404c6776e96)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:17.219155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2c05d-0cdf-4d17-9cd4-28d09f0248c7 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:17.220747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 532/1079] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:55:17 2022] GET /identity/v3/users/1dffeb63ae9242609ce67404c6776e96 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:17.227304 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2c05d-0cdf-4d17-9cd4-28d09f0248c7 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:17.275600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323576d0-9ca8-403a-b707-358911508fa7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:17.311626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe38fae-0a37-47f3-91c5-b50b4ae607d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:17.312236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe38fae-0a37-47f3-91c5-b50b4ae607d7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:17.312519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe38fae-0a37-47f3-91c5-b50b4ae607d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:17.312783 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe38fae-0a37-47f3-91c5-b50b4ae607d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:17.321820 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:17.332001 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2c05d-0cdf-4d17-9cd4-28d09f0248c7 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:17.367621 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f670e4-7ac2-461d-a0e2-76c581981051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:17.368246 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f670e4-7ac2-461d-a0e2-76c581981051 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:17.368518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f670e4-7ac2-461d-a0e2-76c581981051 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:17.368781 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f670e4-7ac2-461d-a0e2-76c581981051 None admin] PATH_INFO: `/v3/projects/e77865d721474450a4f95f9bf7e9f0d1` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:17.369896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f670e4-7ac2-461d-a0e2-76c581981051 None admin] RBAC: Authorizing `identity:get_project(project_id=e77865d721474450a4f95f9bf7e9f0d1)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:17.371360 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f670e4-7ac2-461d-a0e2-76c581981051 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:17.373415 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 533/1080] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:55:17 2022] GET /identity/v3/projects/e77865d721474450a4f95f9bf7e9f0d1 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:17.379649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f670e4-7ac2-461d-a0e2-76c581981051 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:17.386104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe38fae-0a37-47f3-91c5-b50b4ae607d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:17.387590 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe38fae-0a37-47f3-91c5-b50b4ae607d7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:17.438353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:17.486205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f670e4-7ac2-461d-a0e2-76c581981051 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:17.499009 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 548/1081] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:55:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 06 07:55:17.503081 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe38fae-0a37-47f3-91c5-b50b4ae607d7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:17.524376 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2de99-a93a-4b44-a6d8-c84bdb1812af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:17.525577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2de99-a93a-4b44-a6d8-c84bdb1812af None admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:17.526023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2de99-a93a-4b44-a6d8-c84bdb1812af None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:17.526239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2de99-a93a-4b44-a6d8-c84bdb1812af None admin] PATH_INFO: `/v3/projects/e77865d721474450a4f95f9bf7e9f0d1/users/1dffeb63ae9242609ce67404c6776e96/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:17.528675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2de99-a93a-4b44-a6d8-c84bdb1812af None admin] RBAC: Authorizing `identity:create_grant(project_id=e77865d721474450a4f95f9bf7e9f0d1, user_id=1dffeb63ae9242609ce67404c6776e96, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:17.530779 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2de99-a93a-4b44-a6d8-c84bdb1812af None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:17.593192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 534/1082] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:55:17 2022] PUT /identity/v3/projects/e77865d721474450a4f95f9bf7e9f0d1/users/1dffeb63ae9242609ce67404c6776e96/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 07:55:17.623664 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe38fae-0a37-47f3-91c5-b50b4ae607d7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:17.663727 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ef064-b179-4b2b-a4c8-e2644295df06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:17.664370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ef064-b179-4b2b-a4c8-e2644295df06 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:17.664640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ef064-b179-4b2b-a4c8-e2644295df06 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:17.664907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ef064-b179-4b2b-a4c8-e2644295df06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:17.676847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ef064-b179-4b2b-a4c8-e2644295df06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:17.678299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ef064-b179-4b2b-a4c8-e2644295df06 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:17.694566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-f22ef064-b179-4b2b-a4c8-e2644295df06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:55:17.694566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:17.694566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:17.694566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:17.694566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:17.694566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:17.694566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:17.694566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:17.694566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:17.694566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:17.694566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:17.694566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:17.694566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:55:17.694566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:55:17.694566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:55:17.694566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:17.695703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:55:17.696620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:55:17.696620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:55:17.696620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:17.699272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 549/1083] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:55:17.701792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2de99-a93a-4b44-a6d8-c84bdb1812af None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:17.724267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:17.825506 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2de99-a93a-4b44-a6d8-c84bdb1812af None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:17.861397 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195e0a3-c034-4fb3-b732-1ea224390a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:17.862003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195e0a3-c034-4fb3-b732-1ea224390a6b None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:17.862274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195e0a3-c034-4fb3-b732-1ea224390a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:17.863092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195e0a3-c034-4fb3-b732-1ea224390a6b None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:17.863916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195e0a3-c034-4fb3-b732-1ea224390a6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:17.864213 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195e0a3-c034-4fb3-b732-1ea224390a6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:17.865575 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195e0a3-c034-4fb3-b732-1ea224390a6b None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:17.877258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 535/1084] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:18.813551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1503a52-17ac-4cdb-95ca-7ce94092c766 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:18.814103 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1503a52-17ac-4cdb-95ca-7ce94092c766 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:18.814103 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1503a52-17ac-4cdb-95ca-7ce94092c766 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:18.815524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 550/1085] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:18.820863 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecd942-d833-473a-959b-2a2aa825cdfa None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:18.821304 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecd942-d833-473a-959b-2a2aa825cdfa None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:18.821399 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecd942-d833-473a-959b-2a2aa825cdfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:18.839783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-03ecd942-d833-473a-959b-2a2aa825cdfa None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:18.840962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-03ecd942-d833-473a-959b-2a2aa825cdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:18.915573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 536/1086] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:19.194703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1503a52-17ac-4cdb-95ca-7ce94092c766 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:19.197626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:19.296452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1503a52-17ac-4cdb-95ca-7ce94092c766 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:19.329106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3a29b-fe43-4738-bfa9-5177d373eb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:19.329723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3a29b-fe43-4738-bfa9-5177d373eb85 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:19.329994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3a29b-fe43-4738-bfa9-5177d373eb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:19.330259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3a29b-fe43-4738-bfa9-5177d373eb85 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:19.340592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3a29b-fe43-4738-bfa9-5177d373eb85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:55:19.345302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-8de3a29b-fe43-4738-bfa9-5177d373eb85 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 07:55:19.345302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:19.345302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:19.345302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:19.345302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:19.345302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:19.345302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:19.345302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:19.345302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:19.345302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:19.345302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:19.345302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:19.345302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:55:19.345302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:55:19.345302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:55:19.345302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:55:19.346428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:55:19.346428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:55:19.346428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:55:19.346428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:55:19.346428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:19.346428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:19.346428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:19.346428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:19.346428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:19.346428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:19.346428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:19.346428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:19.346428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:19.346428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:19.346428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:19.346428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:19.347891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:19.347891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:19.347891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:19.347891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:19.347891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:55:19.347891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:55:19.347891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:55:19.347891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:55:19.347891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:55:19.347891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:55:19.347891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 06 07:55:19.347891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:19.347891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 551/1087] 10.222.0.30 () {56 vars in 1120 bytes} [Thu Oct 6 07:55:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:19.353157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecd942-d833-473a-959b-2a2aa825cdfa None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:19.363238 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:19.450399 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecd942-d833-473a-959b-2a2aa825cdfa None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:19.490288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2be409-4ab2-4b76-928f-370829c4b311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:19.490896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2be409-4ab2-4b76-928f-370829c4b311 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:19.491199 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2be409-4ab2-4b76-928f-370829c4b311 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:19.491469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2be409-4ab2-4b76-928f-370829c4b311 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:19.492129 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2be409-4ab2-4b76-928f-370829c4b311 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:19.492427 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2be409-4ab2-4b76-928f-370829c4b311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:19.493794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2be409-4ab2-4b76-928f-370829c4b311 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:19.502125 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 537/1088] 10.222.0.30 () {56 vars in 1115 bytes} [Thu Oct 6 07:55:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:19.508100 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3a29b-fe43-4738-bfa9-5177d373eb85 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:19.570291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:19.608906 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3a29b-fe43-4738-bfa9-5177d373eb85 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:19.642240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47392a94-c764-4ad0-ad5b-3d432ea37b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:19.642852 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47392a94-c764-4ad0-ad5b-3d432ea37b90 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:19.643150 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47392a94-c764-4ad0-ad5b-3d432ea37b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:19.643416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47392a94-c764-4ad0-ad5b-3d432ea37b90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:19.654191 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47392a94-c764-4ad0-ad5b-3d432ea37b90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:19.655659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47392a94-c764-4ad0-ad5b-3d432ea37b90 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:19.670060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-47392a94-c764-4ad0-ad5b-3d432ea37b90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:55:19.670060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:19.670060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:19.670060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:19.670060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:19.670060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:19.670060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:19.670060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:19.670060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:19.670060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:19.670060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:19.670060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:19.670060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:55:19.670060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:55:19.670060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:55:19.670060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:19.676352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:55:19.677377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:55:19.677377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:55:19.677377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:19.677377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 552/1089] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:55:19.677659 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2be409-4ab2-4b76-928f-370829c4b311 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:19.749820 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:19.781542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2be409-4ab2-4b76-928f-370829c4b311 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:19.788627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf57a41-2a58-4ce5-9f95-58881e232e4a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:19.788913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf57a41-2a58-4ce5-9f95-58881e232e4a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:19.789182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf57a41-2a58-4ce5-9f95-58881e232e4a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:19.790517 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 553/1090] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:55:19.795511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db350d05-ae4c-4767-9d66-9384c3349788 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:19.795791 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db350d05-ae4c-4767-9d66-9384c3349788 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:19.796055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db350d05-ae4c-4767-9d66-9384c3349788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:19.814476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-db350d05-ae4c-4767-9d66-9384c3349788 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:19.816502 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba10c4-252e-4c2e-a41f-c3cff881a158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:19.816759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-db350d05-ae4c-4767-9d66-9384c3349788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:19.817100 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba10c4-252e-4c2e-a41f-c3cff881a158 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:19.817373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba10c4-252e-4c2e-a41f-c3cff881a158 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:19.817641 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba10c4-252e-4c2e-a41f-c3cff881a158 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:19.818271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba10c4-252e-4c2e-a41f-c3cff881a158 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:19.818571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba10c4-252e-4c2e-a41f-c3cff881a158 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:19.819956 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba10c4-252e-4c2e-a41f-c3cff881a158 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:19.832134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 538/1091] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:19.837697 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba10c4-252e-4c2e-a41f-c3cff881a158 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:19.884725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 554/1092] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:55:19.934242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba10c4-252e-4c2e-a41f-c3cff881a158 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:19.962584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:19.968593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1576852-c22e-465b-8912-28e2f0431626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:19.969213 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1576852-c22e-465b-8912-28e2f0431626 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:19.969484 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1576852-c22e-465b-8912-28e2f0431626 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:19.969749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1576852-c22e-465b-8912-28e2f0431626 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:19.976634 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1576852-c22e-465b-8912-28e2f0431626 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:19.978057 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1576852-c22e-465b-8912-28e2f0431626 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:19.990274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-d1576852-c22e-465b-8912-28e2f0431626 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:55:19.990274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:19.990274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:19.990274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:19.990274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:19.990274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:19.990274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:19.990274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:19.990274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:19.990274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:19.990274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:19.990274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:19.990274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:55:19.990274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:55:19.990274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:55:19.990274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:19.991285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:19.992424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:19.992424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:19.992424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:55:19.992424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:55:19.992424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:55:19.992424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:55:19.992424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:55:19.992424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:55:19.992424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 06 07:55:19.992424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:19.995097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 539/1093] 10.222.0.30 () {56 vars in 1093 bytes} [Thu Oct 6 07:55:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:55:19.997169 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db350d05-ae4c-4767-9d66-9384c3349788 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:20.041710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:20.094324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db350d05-ae4c-4767-9d66-9384c3349788 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:20.127742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542d121-2b3f-4c2c-be2e-65907746be7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:20.129325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542d121-2b3f-4c2c-be2e-65907746be7f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:20.129602 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542d121-2b3f-4c2c-be2e-65907746be7f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:20.129871 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542d121-2b3f-4c2c-be2e-65907746be7f None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:20.130593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542d121-2b3f-4c2c-be2e-65907746be7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:20.130895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542d121-2b3f-4c2c-be2e-65907746be7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:20.132345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542d121-2b3f-4c2c-be2e-65907746be7f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:20.150226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1576852-c22e-465b-8912-28e2f0431626 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:20.153058 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 555/1094] 10.222.0.30 () {56 vars in 1133 bytes} [Thu Oct 6 07:55:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:20.157496 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542d121-2b3f-4c2c-be2e-65907746be7f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:20.251829 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1576852-c22e-465b-8912-28e2f0431626 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:20.255142 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542d121-2b3f-4c2c-be2e-65907746be7f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:20.256651 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:20.287161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110f364-db81-4f6b-9ec7-2ba4772ee025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:20.287782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110f364-db81-4f6b-9ec7-2ba4772ee025 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:20.288059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110f364-db81-4f6b-9ec7-2ba4772ee025 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:20.288326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110f364-db81-4f6b-9ec7-2ba4772ee025 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:20.289756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc114f7-d6aa-4393-acb1-5d82240e7f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:20.290348 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc114f7-d6aa-4393-acb1-5d82240e7f84 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:20.290619 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc114f7-d6aa-4393-acb1-5d82240e7f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:20.290885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc114f7-d6aa-4393-acb1-5d82240e7f84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:20.299669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5110f364-db81-4f6b-9ec7-2ba4772ee025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:55:20.305467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-5110f364-db81-4f6b-9ec7-2ba4772ee025 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:55:20.305467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:20.305467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:20.305467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:20.305467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:20.305467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:20.305467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:20.305467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:20.305467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:20.305467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:20.305467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:20.305467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:20.305467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:55:20.305467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:55:20.305467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:55:20.305467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:55:20.306450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:55:20.306450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:55:20.306450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:55:20.306450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:55:20.306450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:20.306450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:20.306450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:20.306450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:20.306450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:20.306450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:20.306450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:20.306450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:20.306450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:20.306450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:20.306450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:20.306450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:20.307341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:20.307341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:20.307341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:20.307341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:20.307341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:55:20.307341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:55:20.307341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:55:20.307341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:55:20.307341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:55:20.307341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:55:20.307341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:55:20.307341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:20.307341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 540/1095] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:55:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:55:20.312552 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110f364-db81-4f6b-9ec7-2ba4772ee025 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:20.358276 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc114f7-d6aa-4393-acb1-5d82240e7f84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:20.359741 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc114f7-d6aa-4393-acb1-5d82240e7f84 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:20.364055 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:20.414551 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110f364-db81-4f6b-9ec7-2ba4772ee025 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:20.447102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700848b1-e535-4725-8658-99ecf87be81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:20.447693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700848b1-e535-4725-8658-99ecf87be81f None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:20.447967 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700848b1-e535-4725-8658-99ecf87be81f None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:20.448234 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700848b1-e535-4725-8658-99ecf87be81f None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:20.448885 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700848b1-e535-4725-8658-99ecf87be81f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:20.449180 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700848b1-e535-4725-8658-99ecf87be81f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:20.450534 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700848b1-e535-4725-8658-99ecf87be81f None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:20.458076 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 556/1096] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:55:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 06 07:55:20.463193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 541/1097] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:55:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:20.463985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc114f7-d6aa-4393-acb1-5d82240e7f84 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:20.464373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700848b1-e535-4725-8658-99ecf87be81f None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:20.465894 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:20.562504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700848b1-e535-4725-8658-99ecf87be81f None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:20.566029 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc114f7-d6aa-4393-acb1-5d82240e7f84 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:20.575438 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:20.595982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729ff6a-417f-4b63-9975-5a5244c4ec61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:20.597198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729ff6a-417f-4b63-9975-5a5244c4ec61 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:20.597474 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729ff6a-417f-4b63-9975-5a5244c4ec61 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:20.597770 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729ff6a-417f-4b63-9975-5a5244c4ec61 None admin] PATH_INFO: `/v3/users/1dffeb63ae9242609ce67404c6776e96` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:20.598893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729ff6a-417f-4b63-9975-5a5244c4ec61 None admin] RBAC: Authorizing `identity:get_user(user_id=1dffeb63ae9242609ce67404c6776e96)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:20.599562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb24cd-5ee7-4c95-a482-a667eadd76d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:20.600181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb24cd-5ee7-4c95-a482-a667eadd76d7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:20.600412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729ff6a-417f-4b63-9975-5a5244c4ec61 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:20.600507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb24cd-5ee7-4c95-a482-a667eadd76d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:20.600708 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb24cd-5ee7-4c95-a482-a667eadd76d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:20.606606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 542/1098] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:55:20 2022] GET /identity/v3/users/1dffeb63ae9242609ce67404c6776e96 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:20.607684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729ff6a-417f-4b63-9975-5a5244c4ec61 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:20.616787 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb24cd-5ee7-4c95-a482-a667eadd76d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:20.618241 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb24cd-5ee7-4c95-a482-a667eadd76d7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:20.632546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3deb24cd-5ee7-4c95-a482-a667eadd76d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:55:20.632546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:20.632546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:20.632546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:20.632546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:20.632546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:20.632546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:20.632546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:20.632546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:20.632546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:20.632546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:20.632546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:20.632546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:55:20.632546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:55:20.632546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:55:20.632546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:20.633686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:55:20.634767 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:55:20.634767 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:55:20.634767 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:20.638424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 557/1099] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:55:20.639313 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb24cd-5ee7-4c95-a482-a667eadd76d7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:20.693424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:20.706443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729ff6a-417f-4b63-9975-5a5244c4ec61 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:20.739385 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb24cd-5ee7-4c95-a482-a667eadd76d7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:20.743730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede0f4e-6a6f-4316-946d-8c2581f316f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:20.744330 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede0f4e-6a6f-4316-946d-8c2581f316f3 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:20.744607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede0f4e-6a6f-4316-946d-8c2581f316f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:20.744877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede0f4e-6a6f-4316-946d-8c2581f316f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:20.773466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6b1e8-265e-40a0-939f-0db6499e7d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:20.774069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6b1e8-265e-40a0-939f-0db6499e7d4f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:20.774345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6b1e8-265e-40a0-939f-0db6499e7d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:20.774617 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6b1e8-265e-40a0-939f-0db6499e7d4f None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:20.775280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6b1e8-265e-40a0-939f-0db6499e7d4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:20.775578 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6b1e8-265e-40a0-939f-0db6499e7d4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:20.776942 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6b1e8-265e-40a0-939f-0db6499e7d4f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:20.789583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 558/1100] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:55:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:20.794346 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:20.795394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6b1e8-265e-40a0-939f-0db6499e7d4f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:20.813594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede0f4e-6a6f-4316-946d-8c2581f316f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:20.815297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede0f4e-6a6f-4316-946d-8c2581f316f3 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:20.893560 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6b1e8-265e-40a0-939f-0db6499e7d4f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:20.907051 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:20.913523 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 543/1101] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:55:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 06 07:55:20.919306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede0f4e-6a6f-4316-946d-8c2581f316f3 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:20.927895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08264423-781d-4125-a4bc-446c4b3401d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:20.928526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08264423-781d-4125-a4bc-446c4b3401d0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:20.928791 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08264423-781d-4125-a4bc-446c4b3401d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:20.929055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08264423-781d-4125-a4bc-446c4b3401d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:20.936213 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08264423-781d-4125-a4bc-446c4b3401d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:20.938018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08264423-781d-4125-a4bc-446c4b3401d0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:20.953660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-08264423-781d-4125-a4bc-446c4b3401d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:55:20.953660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:20.953660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:20.953660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:20.953660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:20.953660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:20.953660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:20.953660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:20.953660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:20.953660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:20.953660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:20.953660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:20.953660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:55:20.953660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:55:20.953660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:20.954997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:20.956609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:55:20.956609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:55:20.956609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:55:20.956609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:55:20.956609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:55:20.956609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:55:20.956609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:55:20.956609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:20.956609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 559/1102] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:55:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:55:20.960670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08264423-781d-4125-a4bc-446c4b3401d0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:21.002868 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:21.020970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede0f4e-6a6f-4316-946d-8c2581f316f3 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:21.055143 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad62fb-7f38-40b0-b25e-f4467dbfd9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:21.055783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad62fb-7f38-40b0-b25e-f4467dbfd9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:21.056053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad62fb-7f38-40b0-b25e-f4467dbfd9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:21.056377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad62fb-7f38-40b0-b25e-f4467dbfd9b5 None admin] PATH_INFO: `/v3/projects/e77865d721474450a4f95f9bf7e9f0d1` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:21.057483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad62fb-7f38-40b0-b25e-f4467dbfd9b5 None admin] RBAC: Authorizing `identity:get_project(project_id=e77865d721474450a4f95f9bf7e9f0d1)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:21.058500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08264423-781d-4125-a4bc-446c4b3401d0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:21.058914 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad62fb-7f38-40b0-b25e-f4467dbfd9b5 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:21.065677 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 544/1103] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:55:20 2022] GET /identity/v3/projects/e77865d721474450a4f95f9bf7e9f0d1 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:21.066799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad62fb-7f38-40b0-b25e-f4467dbfd9b5 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:21.093423 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7e784-d06c-4075-aaf5-a54f82463f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:21.094021 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7e784-d06c-4075-aaf5-a54f82463f7e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:21.094297 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7e784-d06c-4075-aaf5-a54f82463f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:21.094563 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7e784-d06c-4075-aaf5-a54f82463f7e None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:21.095283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7e784-d06c-4075-aaf5-a54f82463f7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:21.095583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7e784-d06c-4075-aaf5-a54f82463f7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:21.097604 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7e784-d06c-4075-aaf5-a54f82463f7e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:21.109786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 560/1104] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:55:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:21.116132 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7e784-d06c-4075-aaf5-a54f82463f7e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:21.153260 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:21.165793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad62fb-7f38-40b0-b25e-f4467dbfd9b5 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:21.199898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf066f0-af0c-4e64-a19d-25a330b868f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:21.200500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf066f0-af0c-4e64-a19d-25a330b868f0 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:21.200776 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf066f0-af0c-4e64-a19d-25a330b868f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:21.201048 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf066f0-af0c-4e64-a19d-25a330b868f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:21.201767 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf066f0-af0c-4e64-a19d-25a330b868f0 None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=e77865d721474450a4f95f9bf7e9f0d1, user.id=1dffeb63ae9242609ce67404c6776e96) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:21.202074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf066f0-af0c-4e64-a19d-25a330b868f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:21.203551 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf066f0-af0c-4e64-a19d-25a330b868f0 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:21.210125 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 545/1105] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:55:21 2022] GET /identity/v3/role_assignments?user.id=1dffeb63ae9242609ce67404c6776e96&scope.project.id=e77865d721474450a4f95f9bf7e9f0d1&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:21.219297 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7e784-d06c-4075-aaf5-a54f82463f7e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:21.258050 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c4121-0341-4b13-b0ff-f12634c2def8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:21.258656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c4121-0341-4b13-b0ff-f12634c2def8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:21.258928 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c4121-0341-4b13-b0ff-f12634c2def8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:21.259222 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c4121-0341-4b13-b0ff-f12634c2def8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:21.259953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c4121-0341-4b13-b0ff-f12634c2def8 None admin] RBAC: Adding query filter params (role.id=c02e5654263d45f88dd97c056d1f9769, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=209795285384486183cda455fe316bc9) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:21.260257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c4121-0341-4b13-b0ff-f12634c2def8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:21.261707 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c4121-0341-4b13-b0ff-f12634c2def8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:21.268331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 561/1106] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:55:21 2022] GET /identity/v3/role_assignments?user.id=209795285384486183cda455fe316bc9&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=c02e5654263d45f88dd97c056d1f9769 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:23.384861 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089a75d-0a7c-4c14-b2d2-79c7de041a4b None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:23.385377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089a75d-0a7c-4c14-b2d2-79c7de041a4b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:23.385377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089a75d-0a7c-4c14-b2d2-79c7de041a4b None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:23.386737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 546/1107] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:23.391925 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9ce5f-c75a-41a1-ba84-3cf6e2c95c47 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:23.392343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9ce5f-c75a-41a1-ba84-3cf6e2c95c47 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:23.392469 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9ce5f-c75a-41a1-ba84-3cf6e2c95c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:23.411147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a0b9ce5f-c75a-41a1-ba84-3cf6e2c95c47 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:23.412241 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a0b9ce5f-c75a-41a1-ba84-3cf6e2c95c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:23.478976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 562/1108] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:55:23.749093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089a75d-0a7c-4c14-b2d2-79c7de041a4b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:23.763659 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:23.848261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089a75d-0a7c-4c14-b2d2-79c7de041a4b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:23.880670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be401ae2-0ecb-4273-afba-9ecb823926f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:23.881279 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be401ae2-0ecb-4273-afba-9ecb823926f6 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:23.881552 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be401ae2-0ecb-4273-afba-9ecb823926f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:23.881820 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be401ae2-0ecb-4273-afba-9ecb823926f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:23.882902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be401ae2-0ecb-4273-afba-9ecb823926f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:23.884348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be401ae2-0ecb-4273-afba-9ecb823926f6 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:23.885884 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 547/1109] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:23.891747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9ce5f-c75a-41a1-ba84-3cf6e2c95c47 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:23.919992 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:23.989852 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9ce5f-c75a-41a1-ba84-3cf6e2c95c47 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:24.022447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809df54-404e-47ca-b72b-c6d3af143890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:24.022992 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809df54-404e-47ca-b72b-c6d3af143890 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:24.023269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809df54-404e-47ca-b72b-c6d3af143890 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:24.023538 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809df54-404e-47ca-b72b-c6d3af143890 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:24.024187 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809df54-404e-47ca-b72b-c6d3af143890 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:24.025608 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809df54-404e-47ca-b72b-c6d3af143890 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:24.112186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 563/1110] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:55:23 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:55:26.326469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478de05-a0df-427f-bb8f-883000845ed7 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:26.327041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478de05-a0df-427f-bb8f-883000845ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:26.327041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478de05-a0df-427f-bb8f-883000845ed7 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:26.328393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 548/1111] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:26.334180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24301919-c0ae-4adb-8633-eee8627a56f2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:26.334595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24301919-c0ae-4adb-8633-eee8627a56f2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:26.334704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24301919-c0ae-4adb-8633-eee8627a56f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:26.353076 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-24301919-c0ae-4adb-8633-eee8627a56f2 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:26.354235 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-24301919-c0ae-4adb-8633-eee8627a56f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:26.418913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 564/1112] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:26.688674 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478de05-a0df-427f-bb8f-883000845ed7 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:26.766327 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:26.786118 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478de05-a0df-427f-bb8f-883000845ed7 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:26.818116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51affb-f4ef-4992-9c14-7d5642fe9af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:26.818725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51affb-f4ef-4992-9c14-7d5642fe9af1 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:26.819023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51affb-f4ef-4992-9c14-7d5642fe9af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:26.819292 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51affb-f4ef-4992-9c14-7d5642fe9af1 None admin] PATH_INFO: `/v3/roles/d9b2237fd9884cc39771e75bf902e203` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:26.820672 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51affb-f4ef-4992-9c14-7d5642fe9af1 None admin] RBAC: Authorizing `identity:get_role(role_id=d9b2237fd9884cc39771e75bf902e203)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:26.822121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51affb-f4ef-4992-9c14-7d5642fe9af1 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:26.823473 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 549/1113] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:55:26 2022] GET /identity/v3/roles/d9b2237fd9884cc39771e75bf902e203 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:26.829427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24301919-c0ae-4adb-8633-eee8627a56f2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:26.844555 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:26.926718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24301919-c0ae-4adb-8633-eee8627a56f2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:26.965411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3d48a-abaf-434e-9d5f-a1ae03438fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:26.965962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3d48a-abaf-434e-9d5f-a1ae03438fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:26.966239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3d48a-abaf-434e-9d5f-a1ae03438fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:26.966510 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3d48a-abaf-434e-9d5f-a1ae03438fe8 None admin] PATH_INFO: `/v3/users/2f2ea8d8ac3a49deaf478082b24d8b0e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:26.980175 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3d48a-abaf-434e-9d5f-a1ae03438fe8 None admin] RBAC: Authorizing `identity:get_user(user_id=2f2ea8d8ac3a49deaf478082b24d8b0e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:26.981627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3d48a-abaf-434e-9d5f-a1ae03438fe8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:26.983445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 565/1114] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:55:26 2022] GET /identity/v3/users/2f2ea8d8ac3a49deaf478082b24d8b0e => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:26.989752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51affb-f4ef-4992-9c14-7d5642fe9af1 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:27.086198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51affb-f4ef-4992-9c14-7d5642fe9af1 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:27.119570 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed0ada-0a7e-4cec-88e0-90530414190e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:27.120158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed0ada-0a7e-4cec-88e0-90530414190e None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:27.120429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed0ada-0a7e-4cec-88e0-90530414190e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:27.120694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed0ada-0a7e-4cec-88e0-90530414190e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:27.138080 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:27.185240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed0ada-0a7e-4cec-88e0-90530414190e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:27.186677 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed0ada-0a7e-4cec-88e0-90530414190e None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:27.281125 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 550/1115] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:55:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 07:55:27.287236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3d48a-abaf-434e-9d5f-a1ae03438fe8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:27.355488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:27.383903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3d48a-abaf-434e-9d5f-a1ae03438fe8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:27.416379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388848c-d7bb-4ec3-8211-05e9dc5f7c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:27.416988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388848c-d7bb-4ec3-8211-05e9dc5f7c2b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:27.417259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388848c-d7bb-4ec3-8211-05e9dc5f7c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:27.417525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388848c-d7bb-4ec3-8211-05e9dc5f7c2b None admin] PATH_INFO: `/v3/projects/e77865d721474450a4f95f9bf7e9f0d1` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:27.418606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388848c-d7bb-4ec3-8211-05e9dc5f7c2b None admin] RBAC: Authorizing `identity:get_project(project_id=e77865d721474450a4f95f9bf7e9f0d1)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:27.420044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388848c-d7bb-4ec3-8211-05e9dc5f7c2b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:27.421625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 566/1116] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:55:27 2022] GET /identity/v3/projects/e77865d721474450a4f95f9bf7e9f0d1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:27.428148 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed0ada-0a7e-4cec-88e0-90530414190e None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:27.475461 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:27.528194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed0ada-0a7e-4cec-88e0-90530414190e None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:27.560447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc3a13-88e6-458d-85e2-50a393e72976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:27.560985 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc3a13-88e6-458d-85e2-50a393e72976 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:27.561254 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc3a13-88e6-458d-85e2-50a393e72976 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:27.561516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc3a13-88e6-458d-85e2-50a393e72976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:27.562220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc3a13-88e6-458d-85e2-50a393e72976 None admin] RBAC: Adding query filter params (role.id=d9b2237fd9884cc39771e75bf902e203, scope.project.id=e77865d721474450a4f95f9bf7e9f0d1, user.id=2f2ea8d8ac3a49deaf478082b24d8b0e) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:27.562517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc3a13-88e6-458d-85e2-50a393e72976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:27.563954 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc3a13-88e6-458d-85e2-50a393e72976 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:27.569224 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 551/1117] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:55:27 2022] GET /identity/v3/role_assignments?user.id=2f2ea8d8ac3a49deaf478082b24d8b0e&scope.project.id=e77865d721474450a4f95f9bf7e9f0d1&role.id=d9b2237fd9884cc39771e75bf902e203 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:29.669244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ba1ce-51a5-4dba-83ad-0b85c3a3d13e None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:29.669774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ba1ce-51a5-4dba-83ad-0b85c3a3d13e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:29.669774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ba1ce-51a5-4dba-83ad-0b85c3a3d13e None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:29.671172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 567/1118] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:29.676418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30fc05-1a74-4900-a5bc-edb0e511fdba None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:29.676832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30fc05-1a74-4900-a5bc-edb0e511fdba None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:29.676971 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30fc05-1a74-4900-a5bc-edb0e511fdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:29.695458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-ad30fc05-1a74-4900-a5bc-edb0e511fdba None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:29.696641 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-ad30fc05-1a74-4900-a5bc-edb0e511fdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:29.761522 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 552/1119] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:30.032389 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ba1ce-51a5-4dba-83ad-0b85c3a3d13e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:30.132286 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ba1ce-51a5-4dba-83ad-0b85c3a3d13e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:30.165429 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481ff13-4edb-488f-9724-ef22f1940a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:30.166046 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481ff13-4edb-488f-9724-ef22f1940a1c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:30.166320 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481ff13-4edb-488f-9724-ef22f1940a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:30.166589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481ff13-4edb-488f-9724-ef22f1940a1c None admin] PATH_INFO: `/v3/roles/d9b2237fd9884cc39771e75bf902e203` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:30.167987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481ff13-4edb-488f-9724-ef22f1940a1c None admin] RBAC: Authorizing `identity:get_role(role_id=d9b2237fd9884cc39771e75bf902e203)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:30.169348 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481ff13-4edb-488f-9724-ef22f1940a1c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:30.170671 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 568/1120] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:55:30 2022] GET /identity/v3/roles/d9b2237fd9884cc39771e75bf902e203 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:30.177205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30fc05-1a74-4900-a5bc-edb0e511fdba None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:30.217211 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:30.271369 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30fc05-1a74-4900-a5bc-edb0e511fdba None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:30.304398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b6e61-099a-4998-9c7c-9da3b785fcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:30.304953 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b6e61-099a-4998-9c7c-9da3b785fcc1 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:30.305226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b6e61-099a-4998-9c7c-9da3b785fcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:30.305490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b6e61-099a-4998-9c7c-9da3b785fcc1 None admin] PATH_INFO: `/v3/users/2f2ea8d8ac3a49deaf478082b24d8b0e` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:30.306581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b6e61-099a-4998-9c7c-9da3b785fcc1 None admin] RBAC: Authorizing `identity:get_user(user_id=2f2ea8d8ac3a49deaf478082b24d8b0e)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:30.308041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b6e61-099a-4998-9c7c-9da3b785fcc1 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:30.309584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 553/1121] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:55:30 2022] GET /identity/v3/users/2f2ea8d8ac3a49deaf478082b24d8b0e => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:30.316017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481ff13-4edb-488f-9724-ef22f1940a1c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:30.326915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:30.412160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481ff13-4edb-488f-9724-ef22f1940a1c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:30.444251 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7477d2a-cdc6-4896-a00e-10a1cbff0498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:30.444854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7477d2a-cdc6-4896-a00e-10a1cbff0498 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:30.445126 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7477d2a-cdc6-4896-a00e-10a1cbff0498 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:30.445390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7477d2a-cdc6-4896-a00e-10a1cbff0498 None admin] PATH_INFO: `/v3/projects/e77865d721474450a4f95f9bf7e9f0d1` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:30.446472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7477d2a-cdc6-4896-a00e-10a1cbff0498 None admin] RBAC: Authorizing `identity:get_project(project_id=e77865d721474450a4f95f9bf7e9f0d1)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:30.447904 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7477d2a-cdc6-4896-a00e-10a1cbff0498 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:30.449494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 569/1122] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:55:30 2022] GET /identity/v3/projects/e77865d721474450a4f95f9bf7e9f0d1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:30.455520 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b6e61-099a-4998-9c7c-9da3b785fcc1 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:30.561782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b6e61-099a-4998-9c7c-9da3b785fcc1 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:30.576530 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:30.598604 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a6716-ce37-40d7-a91d-e96309cf19f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:30.599280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a6716-ce37-40d7-a91d-e96309cf19f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:30.599554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a6716-ce37-40d7-a91d-e96309cf19f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:30.599818 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a6716-ce37-40d7-a91d-e96309cf19f2 None admin] PATH_INFO: `/v3/projects/e77865d721474450a4f95f9bf7e9f0d1/users/2f2ea8d8ac3a49deaf478082b24d8b0e/roles/d9b2237fd9884cc39771e75bf902e203` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:30.601909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a6716-ce37-40d7-a91d-e96309cf19f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e77865d721474450a4f95f9bf7e9f0d1, user_id=2f2ea8d8ac3a49deaf478082b24d8b0e, role_id=d9b2237fd9884cc39771e75bf902e203)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:30.603368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a6716-ce37-40d7-a91d-e96309cf19f2 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:30.665564 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 554/1123] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:55:30 2022] PUT /identity/v3/projects/e77865d721474450a4f95f9bf7e9f0d1/users/2f2ea8d8ac3a49deaf478082b24d8b0e/roles/d9b2237fd9884cc39771e75bf902e203 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:55:32.806917 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c926318-957d-4254-8cd8-35b4b6c68c10 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:32.807466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c926318-957d-4254-8cd8-35b4b6c68c10 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:32.807466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c926318-957d-4254-8cd8-35b4b6c68c10 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:32.808808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 570/1124] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:32.813934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5354e0-ea83-4b83-b52c-e1046da3d255 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:32.814316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5354e0-ea83-4b83-b52c-e1046da3d255 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:32.814473 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5354e0-ea83-4b83-b52c-e1046da3d255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:32.832640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-ea5354e0-ea83-4b83-b52c-e1046da3d255 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:32.833801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-ea5354e0-ea83-4b83-b52c-e1046da3d255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:32.897107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 555/1125] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:33.174887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c926318-957d-4254-8cd8-35b4b6c68c10 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:33.250367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:33.282958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c926318-957d-4254-8cd8-35b4b6c68c10 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:33.314385 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2ecd0-bdb2-45e0-ac19-ce658dc4cb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:33.315548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2ecd0-bdb2-45e0-ac19-ce658dc4cb99 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:33.315817 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2ecd0-bdb2-45e0-ac19-ce658dc4cb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:33.316081 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2ecd0-bdb2-45e0-ac19-ce658dc4cb99 None admin] PATH_INFO: `/v3/roles/d9b2237fd9884cc39771e75bf902e203` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:33.317447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2ecd0-bdb2-45e0-ac19-ce658dc4cb99 None admin] RBAC: Authorizing `identity:get_role(role_id=d9b2237fd9884cc39771e75bf902e203)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:33.318793 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2ecd0-bdb2-45e0-ac19-ce658dc4cb99 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:33.320127 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 571/1126] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:55:33 2022] GET /identity/v3/roles/d9b2237fd9884cc39771e75bf902e203 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:33.325922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5354e0-ea83-4b83-b52c-e1046da3d255 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:33.417422 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:33.420214 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5354e0-ea83-4b83-b52c-e1046da3d255 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:33.452785 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ec4e7-3f60-4a6d-abba-340fb23d239d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:33.453392 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ec4e7-3f60-4a6d-abba-340fb23d239d None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:33.453664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ec4e7-3f60-4a6d-abba-340fb23d239d None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:33.453932 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ec4e7-3f60-4a6d-abba-340fb23d239d None admin] PATH_INFO: `/v3/users/2f2ea8d8ac3a49deaf478082b24d8b0e` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:33.455056 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ec4e7-3f60-4a6d-abba-340fb23d239d None admin] RBAC: Authorizing `identity:get_user(user_id=2f2ea8d8ac3a49deaf478082b24d8b0e)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:33.457142 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ec4e7-3f60-4a6d-abba-340fb23d239d None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:33.460266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 556/1127] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:55:33 2022] GET /identity/v3/users/2f2ea8d8ac3a49deaf478082b24d8b0e => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:33.464341 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2ecd0-bdb2-45e0-ac19-ce658dc4cb99 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:33.562555 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2ecd0-bdb2-45e0-ac19-ce658dc4cb99 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:33.594540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0dc26-9049-4f32-a341-67c7eda7d0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:33.595149 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0dc26-9049-4f32-a341-67c7eda7d0a7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:33.595423 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0dc26-9049-4f32-a341-67c7eda7d0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:33.595686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0dc26-9049-4f32-a341-67c7eda7d0a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:33.609681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:33.661897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0dc26-9049-4f32-a341-67c7eda7d0a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:33.663372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0dc26-9049-4f32-a341-67c7eda7d0a7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:33.756546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 572/1128] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:55:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 07:55:33.762434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ec4e7-3f60-4a6d-abba-340fb23d239d None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:33.857425 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ec4e7-3f60-4a6d-abba-340fb23d239d None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:33.889966 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c3ee3-0935-4651-b32a-58eaf0c0361b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:33.890638 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c3ee3-0935-4651-b32a-58eaf0c0361b None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:33.890966 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c3ee3-0935-4651-b32a-58eaf0c0361b None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:33.891236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c3ee3-0935-4651-b32a-58eaf0c0361b None admin] PATH_INFO: `/v3/projects/e77865d721474450a4f95f9bf7e9f0d1` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:33.892362 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c3ee3-0935-4651-b32a-58eaf0c0361b None admin] RBAC: Authorizing `identity:get_project(project_id=e77865d721474450a4f95f9bf7e9f0d1)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:33.894430 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c3ee3-0935-4651-b32a-58eaf0c0361b None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:33.896053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 557/1129] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:55:33 2022] GET /identity/v3/projects/e77865d721474450a4f95f9bf7e9f0d1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:33.902106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0dc26-9049-4f32-a341-67c7eda7d0a7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:33.945594 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:33.997864 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0dc26-9049-4f32-a341-67c7eda7d0a7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:34.031290 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99bb13-cca5-483b-9752-e6483e119a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:34.031825 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99bb13-cca5-483b-9752-e6483e119a82 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:34.032096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99bb13-cca5-483b-9752-e6483e119a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:34.032362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99bb13-cca5-483b-9752-e6483e119a82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:34.033076 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99bb13-cca5-483b-9752-e6483e119a82 None admin] RBAC: Adding query filter params (role.id=d9b2237fd9884cc39771e75bf902e203, scope.project.id=e77865d721474450a4f95f9bf7e9f0d1, user.id=2f2ea8d8ac3a49deaf478082b24d8b0e) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:34.033865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99bb13-cca5-483b-9752-e6483e119a82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:34.035431 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99bb13-cca5-483b-9752-e6483e119a82 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:34.040947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 573/1130] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:55:33 2022] GET /identity/v3/role_assignments?user.id=2f2ea8d8ac3a49deaf478082b24d8b0e&scope.project.id=e77865d721474450a4f95f9bf7e9f0d1&role.id=d9b2237fd9884cc39771e75bf902e203 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:36.202238 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5870741-1798-4c76-ad3c-ef03b14dfde4 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:36.202719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5870741-1798-4c76-ad3c-ef03b14dfde4 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:36.202858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5870741-1798-4c76-ad3c-ef03b14dfde4 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:36.204151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 558/1131] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:36.209531 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21be3f4-436a-4a25-b4bc-85bcfc51f6d0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:36.209933 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21be3f4-436a-4a25-b4bc-85bcfc51f6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:36.210064 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21be3f4-436a-4a25-b4bc-85bcfc51f6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:36.228658 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c21be3f4-436a-4a25-b4bc-85bcfc51f6d0 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:36.229847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c21be3f4-436a-4a25-b4bc-85bcfc51f6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:36.293629 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 574/1132] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:36.559797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5870741-1798-4c76-ad3c-ef03b14dfde4 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:36.594551 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:36.659032 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5870741-1798-4c76-ad3c-ef03b14dfde4 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:36.691499 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded98435-cf0d-4954-801d-6d139779a5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:36.692117 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded98435-cf0d-4954-801d-6d139779a5fa None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:36.692393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded98435-cf0d-4954-801d-6d139779a5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:36.692662 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded98435-cf0d-4954-801d-6d139779a5fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:36.693764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded98435-cf0d-4954-801d-6d139779a5fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:36.695216 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded98435-cf0d-4954-801d-6d139779a5fa None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:36.696768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 559/1133] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:36.702462 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21be3f4-436a-4a25-b4bc-85bcfc51f6d0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:36.740909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:36.798732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21be3f4-436a-4a25-b4bc-85bcfc51f6d0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:36.831556 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58f0e1-4853-484e-ac72-05b7afe8031a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:36.832086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58f0e1-4853-484e-ac72-05b7afe8031a None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:36.832362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58f0e1-4853-484e-ac72-05b7afe8031a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:36.832631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58f0e1-4853-484e-ac72-05b7afe8031a None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:36.833279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58f0e1-4853-484e-ac72-05b7afe8031a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:36.834706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58f0e1-4853-484e-ac72-05b7afe8031a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:36.929973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 575/1134] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Oct 6 07:55:36 2022] POST /identity/v3/projects => generated 311 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:55:39.123353 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7750732-9128-4056-ac63-d08d6a0b5d3d None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:39.123924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7750732-9128-4056-ac63-d08d6a0b5d3d None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:39.123924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7750732-9128-4056-ac63-d08d6a0b5d3d None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:39.125251 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 560/1135] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:39.131122 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b06e9-e94c-46a5-b8bf-5064526950f9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:39.131516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b06e9-e94c-46a5-b8bf-5064526950f9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:39.131677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b06e9-e94c-46a5-b8bf-5064526950f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:39.149769 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cb2b06e9-e94c-46a5-b8bf-5064526950f9 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:39.150979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cb2b06e9-e94c-46a5-b8bf-5064526950f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:39.216967 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 576/1136] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:39.483561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7750732-9128-4056-ac63-d08d6a0b5d3d None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:39.584485 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7750732-9128-4056-ac63-d08d6a0b5d3d None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:39.586755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:39.616952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae5b7e-df4f-43d7-990f-c2b2f23d96e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:39.617561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae5b7e-df4f-43d7-990f-c2b2f23d96e9 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:39.617832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae5b7e-df4f-43d7-990f-c2b2f23d96e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:39.618099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae5b7e-df4f-43d7-990f-c2b2f23d96e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:39.619212 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae5b7e-df4f-43d7-990f-c2b2f23d96e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:39.620643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae5b7e-df4f-43d7-990f-c2b2f23d96e9 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:39.622189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 561/1137] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:55:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:39.628286 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b06e9-e94c-46a5-b8bf-5064526950f9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:39.648756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:39.724027 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b06e9-e94c-46a5-b8bf-5064526950f9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:39.755913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65126ba9-f3d1-459c-bc1c-756ef814c91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:39.756494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65126ba9-f3d1-459c-bc1c-756ef814c91e None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:39.756767 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65126ba9-f3d1-459c-bc1c-756ef814c91e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:39.757035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65126ba9-f3d1-459c-bc1c-756ef814c91e None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:39.757685 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65126ba9-f3d1-459c-bc1c-756ef814c91e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:39.759127 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65126ba9-f3d1-459c-bc1c-756ef814c91e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:39.831167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 577/1138] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:55:39 2022] POST /identity/v3/users => generated 286 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:55:42.009993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc084376-faa2-4fe7-9945-6eb2ea3870b4 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:42.010528 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc084376-faa2-4fe7-9945-6eb2ea3870b4 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:42.010528 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc084376-faa2-4fe7-9945-6eb2ea3870b4 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:42.011898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 562/1139] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:42.016995 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92525b74-830c-4706-b414-1247b5c27421 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:42.017991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92525b74-830c-4706-b414-1247b5c27421 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:42.018576 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92525b74-830c-4706-b414-1247b5c27421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:42.036972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-92525b74-830c-4706-b414-1247b5c27421 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:42.038161 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-92525b74-830c-4706-b414-1247b5c27421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:42.103131 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 578/1140] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:42.376883 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc084376-faa2-4fe7-9945-6eb2ea3870b4 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:42.474862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc084376-faa2-4fe7-9945-6eb2ea3870b4 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:42.509091 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15aca4-5c9c-49ad-95ff-143fa28301c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:42.509696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15aca4-5c9c-49ad-95ff-143fa28301c4 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:42.509971 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15aca4-5c9c-49ad-95ff-143fa28301c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:42.510239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15aca4-5c9c-49ad-95ff-143fa28301c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:42.520558 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15aca4-5c9c-49ad-95ff-143fa28301c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:55:42.525228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-4b15aca4-5c9c-49ad-95ff-143fa28301c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:55:42.525228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:42.525228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:42.525228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:42.525228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:42.525228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:42.525228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:42.525228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:42.525228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:42.525228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:42.525228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:42.525228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:42.525228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:55:42.525228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:55:42.525228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:55:42.525228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:55:42.526311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:55:42.526311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:55:42.526311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:55:42.526311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:55:42.526311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:42.526311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:42.526311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:42.526311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:42.526311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:42.526311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:42.526311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:42.526311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:42.526311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:42.526311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:42.526311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:42.526311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:42.527550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:42.527550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:42.527550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:42.527550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:42.527550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:55:42.527550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:55:42.527550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:55:42.527550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:55:42.527550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:55:42.527550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:55:42.527550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:55:42.527550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:42.527550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 563/1141] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:55:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:42.532412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92525b74-830c-4706-b414-1247b5c27421 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:42.541513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:42.630895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92525b74-830c-4706-b414-1247b5c27421 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:42.664086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedb8e1-97e3-4acf-b02d-0c58efd17d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:42.664669 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedb8e1-97e3-4acf-b02d-0c58efd17d09 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:42.664941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedb8e1-97e3-4acf-b02d-0c58efd17d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:42.665208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedb8e1-97e3-4acf-b02d-0c58efd17d09 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:42.665859 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daedb8e1-97e3-4acf-b02d-0c58efd17d09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:42.666154 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daedb8e1-97e3-4acf-b02d-0c58efd17d09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:42.667532 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daedb8e1-97e3-4acf-b02d-0c58efd17d09 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:42.675972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 579/1142] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:55:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:42.681898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15aca4-5c9c-49ad-95ff-143fa28301c4 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:42.713039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:42.779446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15aca4-5c9c-49ad-95ff-143fa28301c4 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:42.811934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9e16a-512d-416e-a6c7-3b6a08bbd131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:42.812540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e16a-512d-416e-a6c7-3b6a08bbd131 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:42.812812 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e16a-512d-416e-a6c7-3b6a08bbd131 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:42.813079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e16a-512d-416e-a6c7-3b6a08bbd131 None admin] PATH_INFO: `/v3/users/9cbea9b21081436fb5423635fe59df6e` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:42.826213 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9e16a-512d-416e-a6c7-3b6a08bbd131 None admin] RBAC: Authorizing `identity:get_user(user_id=9cbea9b21081436fb5423635fe59df6e)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:42.827655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9e16a-512d-416e-a6c7-3b6a08bbd131 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:42.829216 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 564/1143] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:55:42 2022] GET /identity/v3/users/9cbea9b21081436fb5423635fe59df6e => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:42.835334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedb8e1-97e3-4acf-b02d-0c58efd17d09 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:42.898607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:42.931463 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedb8e1-97e3-4acf-b02d-0c58efd17d09 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:42.963840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0afcf1-09cf-45ce-9683-47f69a741f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:42.964430 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0afcf1-09cf-45ce-9683-47f69a741f48 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:42.964708 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0afcf1-09cf-45ce-9683-47f69a741f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:42.964979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0afcf1-09cf-45ce-9683-47f69a741f48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:43.033376 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0afcf1-09cf-45ce-9683-47f69a741f48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:43.034915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0afcf1-09cf-45ce-9683-47f69a741f48 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:43.100051 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:43.133891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 580/1144] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:55:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 07:55:43.139816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9e16a-512d-416e-a6c7-3b6a08bbd131 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:43.227263 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:43.237597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9e16a-512d-416e-a6c7-3b6a08bbd131 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:43.271104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a38a1-69df-464d-87e3-a3c4b287425c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:43.271713 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a38a1-69df-464d-87e3-a3c4b287425c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:43.271985 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a38a1-69df-464d-87e3-a3c4b287425c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:43.272251 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a38a1-69df-464d-87e3-a3c4b287425c None admin] PATH_INFO: `/v3/projects/03c7e1bd6053414686e61e72dd44af71` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:43.273335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a38a1-69df-464d-87e3-a3c4b287425c None admin] RBAC: Authorizing `identity:get_project(project_id=03c7e1bd6053414686e61e72dd44af71)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:43.274751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a38a1-69df-464d-87e3-a3c4b287425c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:43.276388 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 565/1145] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:55:43 2022] GET /identity/v3/projects/03c7e1bd6053414686e61e72dd44af71 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:43.282526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0afcf1-09cf-45ce-9683-47f69a741f48 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:43.378270 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0afcf1-09cf-45ce-9683-47f69a741f48 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:43.412253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3da23-25a4-4829-bf9f-496435360f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:43.412781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3da23-25a4-4829-bf9f-496435360f9f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:43.413059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3da23-25a4-4829-bf9f-496435360f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:43.413329 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3da23-25a4-4829-bf9f-496435360f9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:43.414044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3da23-25a4-4829-bf9f-496435360f9f None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=03c7e1bd6053414686e61e72dd44af71, user.id=9cbea9b21081436fb5423635fe59df6e) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:43.414350 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3da23-25a4-4829-bf9f-496435360f9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:43.415794 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3da23-25a4-4829-bf9f-496435360f9f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:43.420998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 581/1146] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:55:43 2022] GET /identity/v3/role_assignments?user.id=9cbea9b21081436fb5423635fe59df6e&scope.project.id=03c7e1bd6053414686e61e72dd44af71&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:45.538967 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7cbed-bbfe-45d8-a0ea-194f1c701795 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:45.539564 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7cbed-bbfe-45d8-a0ea-194f1c701795 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:45.539564 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7cbed-bbfe-45d8-a0ea-194f1c701795 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:45.540877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 566/1147] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:45.545920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f5968-c885-4b20-b6e7-4a5741dd0340 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:45.546366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f5968-c885-4b20-b6e7-4a5741dd0340 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:45.546463 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f5968-c885-4b20-b6e7-4a5741dd0340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:45.565377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-be7f5968-c885-4b20-b6e7-4a5741dd0340 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:45.566547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-be7f5968-c885-4b20-b6e7-4a5741dd0340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:45.590901 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:45.631712 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 582/1148] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:45.898137 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7cbed-bbfe-45d8-a0ea-194f1c701795 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:46.002646 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7cbed-bbfe-45d8-a0ea-194f1c701795 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:46.036100 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333aa9e-38f4-49f2-a4ad-1b1e2ec75ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:46.036713 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333aa9e-38f4-49f2-a4ad-1b1e2ec75ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:46.036987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333aa9e-38f4-49f2-a4ad-1b1e2ec75ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:46.037257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333aa9e-38f4-49f2-a4ad-1b1e2ec75ac6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:46.047018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333aa9e-38f4-49f2-a4ad-1b1e2ec75ac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:55:46.052398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-3333aa9e-38f4-49f2-a4ad-1b1e2ec75ac6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:55:46.052398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:46.052398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:46.052398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:46.052398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:46.052398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:46.052398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:46.052398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:46.052398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:46.052398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:46.052398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:46.052398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:46.052398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:55:46.052398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:55:46.052398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:55:46.052398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:55:46.053401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:55:46.053401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:55:46.053401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:55:46.053401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:55:46.053401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:46.053401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:46.053401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:46.053401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:46.053401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:46.053401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:46.053401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:46.053401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:46.053401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:46.053401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:46.053401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:46.053401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:46.054426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:46.054426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:46.054426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:46.054426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:46.054426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:55:46.054426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:55:46.054426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:55:46.054426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:55:46.054426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:55:46.054426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:55:46.054426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:55:46.054426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:46.054426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 567/1149] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:55:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:46.059703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f5968-c885-4b20-b6e7-4a5741dd0340 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:46.154848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f5968-c885-4b20-b6e7-4a5741dd0340 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:46.186678 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613957b0-ba4b-4764-a094-f4e76326b795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:46.187300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613957b0-ba4b-4764-a094-f4e76326b795 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:46.187574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613957b0-ba4b-4764-a094-f4e76326b795 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:46.187840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613957b0-ba4b-4764-a094-f4e76326b795 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:46.188487 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613957b0-ba4b-4764-a094-f4e76326b795 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:46.188780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613957b0-ba4b-4764-a094-f4e76326b795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:46.190657 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613957b0-ba4b-4764-a094-f4e76326b795 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:46.198280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 583/1150] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:55:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:46.204107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333aa9e-38f4-49f2-a4ad-1b1e2ec75ac6 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:46.223703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:46.302498 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333aa9e-38f4-49f2-a4ad-1b1e2ec75ac6 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:46.341471 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f99146-7f17-403f-aa21-b833068f08fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:46.342105 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f99146-7f17-403f-aa21-b833068f08fb None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:46.342383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f99146-7f17-403f-aa21-b833068f08fb None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:46.342657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f99146-7f17-403f-aa21-b833068f08fb None admin] PATH_INFO: `/v3/users/9cbea9b21081436fb5423635fe59df6e` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:46.343813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f99146-7f17-403f-aa21-b833068f08fb None admin] RBAC: Authorizing `identity:get_user(user_id=9cbea9b21081436fb5423635fe59df6e)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:46.345296 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f99146-7f17-403f-aa21-b833068f08fb None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:46.346997 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 568/1151] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:55:46 2022] GET /identity/v3/users/9cbea9b21081436fb5423635fe59df6e => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:46.352701 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613957b0-ba4b-4764-a094-f4e76326b795 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:46.378637 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:46.449650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613957b0-ba4b-4764-a094-f4e76326b795 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:46.482815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599d039-9e13-439a-9fac-efc52dea6c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:46.483474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599d039-9e13-439a-9fac-efc52dea6c72 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:46.483753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599d039-9e13-439a-9fac-efc52dea6c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:46.484027 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599d039-9e13-439a-9fac-efc52dea6c72 None admin] PATH_INFO: `/v3/projects/03c7e1bd6053414686e61e72dd44af71` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:46.485155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599d039-9e13-439a-9fac-efc52dea6c72 None admin] RBAC: Authorizing `identity:get_project(project_id=03c7e1bd6053414686e61e72dd44af71)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:46.486671 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599d039-9e13-439a-9fac-efc52dea6c72 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:46.488321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 584/1152] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:55:46 2022] GET /identity/v3/projects/03c7e1bd6053414686e61e72dd44af71 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:46.494065 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f99146-7f17-403f-aa21-b833068f08fb None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:46.576441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:46.590892 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f99146-7f17-403f-aa21-b833068f08fb None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:46.624790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598b242-9ac7-4497-b510-7deb37ed36ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:46.625426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598b242-9ac7-4497-b510-7deb37ed36ef None admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:46.625702 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598b242-9ac7-4497-b510-7deb37ed36ef None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:46.625973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598b242-9ac7-4497-b510-7deb37ed36ef None admin] PATH_INFO: `/v3/projects/03c7e1bd6053414686e61e72dd44af71/users/9cbea9b21081436fb5423635fe59df6e/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:46.628588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598b242-9ac7-4497-b510-7deb37ed36ef None admin] RBAC: Authorizing `identity:create_grant(project_id=03c7e1bd6053414686e61e72dd44af71, user_id=9cbea9b21081436fb5423635fe59df6e, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:46.630023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598b242-9ac7-4497-b510-7deb37ed36ef None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:46.739828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 569/1153] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:55:46 2022] PUT /identity/v3/projects/03c7e1bd6053414686e61e72dd44af71/users/9cbea9b21081436fb5423635fe59df6e/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:55:48.873709 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fd66a-2359-43d3-a083-e87707b1d9e6 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:48.874265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fd66a-2359-43d3-a083-e87707b1d9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:48.874265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fd66a-2359-43d3-a083-e87707b1d9e6 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:48.875604 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 585/1154] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:48.881469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9e973-3d39-4f9c-a70e-ad9e3f2f111e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:48.881857 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9e973-3d39-4f9c-a70e-ad9e3f2f111e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:48.882049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9e973-3d39-4f9c-a70e-ad9e3f2f111e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:48.899819 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-a0d9e973-3d39-4f9c-a70e-ad9e3f2f111e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:48.901008 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a0d9e973-3d39-4f9c-a70e-ad9e3f2f111e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:48.964873 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 570/1155] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:49.242061 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fd66a-2359-43d3-a083-e87707b1d9e6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:49.330517 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:49.339646 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fd66a-2359-43d3-a083-e87707b1d9e6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:49.373672 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2b23b-1d00-4bc5-8756-f320060c6fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:49.374287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2b23b-1d00-4bc5-8756-f320060c6fba None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:49.374564 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2b23b-1d00-4bc5-8756-f320060c6fba None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:49.374834 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2b23b-1d00-4bc5-8756-f320060c6fba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:49.384991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2b23b-1d00-4bc5-8756-f320060c6fba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:55:49.389753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-68c2b23b-1d00-4bc5-8756-f320060c6fba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:55:49.389753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:49.389753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:49.389753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:49.389753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:49.389753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:49.389753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:49.389753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:49.389753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:49.389753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:49.389753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:49.389753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:49.389753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:55:49.389753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:55:49.389753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:55:49.389753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:55:49.390849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:55:49.390849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:55:49.390849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:55:49.390849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:55:49.390849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:49.390849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:49.390849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:49.390849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:49.390849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:49.390849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:49.390849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:49.390849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:49.390849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:49.390849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:49.390849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:49.390849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:49.392472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:49.392472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:49.392472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:49.392472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:49.392472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:55:49.392472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:55:49.392472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:55:49.392472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:55:49.392472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:55:49.392472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:55:49.392472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:55:49.392472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:55:49.392472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 586/1156] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:55:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:49.397680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9e973-3d39-4f9c-a70e-ad9e3f2f111e None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:49.472121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:49.493595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9e973-3d39-4f9c-a70e-ad9e3f2f111e None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:49.526813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ceb158-39b2-4147-be0a-b0406085de74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:49.527444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ceb158-39b2-4147-be0a-b0406085de74 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:49.527907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ceb158-39b2-4147-be0a-b0406085de74 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:49.528397 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ceb158-39b2-4147-be0a-b0406085de74 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:49.529477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ceb158-39b2-4147-be0a-b0406085de74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:49.529791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ceb158-39b2-4147-be0a-b0406085de74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:49.531221 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ceb158-39b2-4147-be0a-b0406085de74 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:49.539818 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 571/1157] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:55:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:49.545659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2b23b-1d00-4bc5-8756-f320060c6fba None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:49.642600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2b23b-1d00-4bc5-8756-f320060c6fba None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:49.677574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da873d6-7807-40f4-b2b7-61b3ec0c140f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:49.678187 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da873d6-7807-40f4-b2b7-61b3ec0c140f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:49.678456 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da873d6-7807-40f4-b2b7-61b3ec0c140f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:49.678718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da873d6-7807-40f4-b2b7-61b3ec0c140f None admin] PATH_INFO: `/v3/users/9cbea9b21081436fb5423635fe59df6e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:49.679835 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da873d6-7807-40f4-b2b7-61b3ec0c140f None admin] RBAC: Authorizing `identity:get_user(user_id=9cbea9b21081436fb5423635fe59df6e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:49.681251 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da873d6-7807-40f4-b2b7-61b3ec0c140f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:49.682817 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 587/1158] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:55:49 2022] GET /identity/v3/users/9cbea9b21081436fb5423635fe59df6e => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:49.688603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ceb158-39b2-4147-be0a-b0406085de74 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:49.787392 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ceb158-39b2-4147-be0a-b0406085de74 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:49.819412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce96de-1bc0-4392-8d97-c0a2b9aa149e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:49.820030 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce96de-1bc0-4392-8d97-c0a2b9aa149e None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:49.820301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce96de-1bc0-4392-8d97-c0a2b9aa149e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:49.820567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce96de-1bc0-4392-8d97-c0a2b9aa149e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:49.828586 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:49.885622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce96de-1bc0-4392-8d97-c0a2b9aa149e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:49.887101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce96de-1bc0-4392-8d97-c0a2b9aa149e None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:49.990088 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 572/1159] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:55:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 07:55:49.996199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da873d6-7807-40f4-b2b7-61b3ec0c140f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:50.018388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:50.093141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da873d6-7807-40f4-b2b7-61b3ec0c140f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:50.127101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5243c7-f185-45a4-b40d-9f71ff54dc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:50.127710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5243c7-f185-45a4-b40d-9f71ff54dc21 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:50.127986 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5243c7-f185-45a4-b40d-9f71ff54dc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:50.128254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5243c7-f185-45a4-b40d-9f71ff54dc21 None admin] PATH_INFO: `/v3/projects/03c7e1bd6053414686e61e72dd44af71` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:50.129363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5243c7-f185-45a4-b40d-9f71ff54dc21 None admin] RBAC: Authorizing `identity:get_project(project_id=03c7e1bd6053414686e61e72dd44af71)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:50.130781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5243c7-f185-45a4-b40d-9f71ff54dc21 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:50.132424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 588/1160] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:55:49 2022] GET /identity/v3/projects/03c7e1bd6053414686e61e72dd44af71 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:50.138647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce96de-1bc0-4392-8d97-c0a2b9aa149e None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:50.176778 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:50.233752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce96de-1bc0-4392-8d97-c0a2b9aa149e None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:50.274177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d897a8-49c2-4f49-8af0-2eb2d289a116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:50.274834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d897a8-49c2-4f49-8af0-2eb2d289a116 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:50.275167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d897a8-49c2-4f49-8af0-2eb2d289a116 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:50.275433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d897a8-49c2-4f49-8af0-2eb2d289a116 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:50.276223 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d897a8-49c2-4f49-8af0-2eb2d289a116 None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=03c7e1bd6053414686e61e72dd44af71, user.id=9cbea9b21081436fb5423635fe59df6e) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:50.276542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d897a8-49c2-4f49-8af0-2eb2d289a116 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:50.277942 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d897a8-49c2-4f49-8af0-2eb2d289a116 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:50.284230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 573/1161] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:55:50 2022] GET /identity/v3/role_assignments?user.id=9cbea9b21081436fb5423635fe59df6e&scope.project.id=03c7e1bd6053414686e61e72dd44af71&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:52.481602 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c2a47-3bce-4ffb-ade3-b3246c1b8272 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:52.482123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c2a47-3bce-4ffb-ade3-b3246c1b8272 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:52.482123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c2a47-3bce-4ffb-ade3-b3246c1b8272 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:52.483439 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 589/1162] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:52.489238 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e532b-4e32-4d8f-a9dc-f4f0bb8e3b48 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:52.489633 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e532b-4e32-4d8f-a9dc-f4f0bb8e3b48 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:52.489779 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e532b-4e32-4d8f-a9dc-f4f0bb8e3b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:52.507948 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-a90e532b-4e32-4d8f-a9dc-f4f0bb8e3b48 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:52.509112 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a90e532b-4e32-4d8f-a9dc-f4f0bb8e3b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:52.573033 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 574/1163] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:52.838189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c2a47-3bce-4ffb-ade3-b3246c1b8272 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:52.923996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:52.936086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c2a47-3bce-4ffb-ade3-b3246c1b8272 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:52.967625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067df47-d457-4b0f-ae36-6a3f96296e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:52.968207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067df47-d457-4b0f-ae36-6a3f96296e1e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:52.968478 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067df47-d457-4b0f-ae36-6a3f96296e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:52.968744 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067df47-d457-4b0f-ae36-6a3f96296e1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:53.033644 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067df47-d457-4b0f-ae36-6a3f96296e1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:53.035100 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067df47-d457-4b0f-ae36-6a3f96296e1e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:53.117638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:53.129806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 590/1164] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:55:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 07:55:53.135712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e532b-4e32-4d8f-a9dc-f4f0bb8e3b48 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:53.171107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:53.242032 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e532b-4e32-4d8f-a9dc-f4f0bb8e3b48 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:53.275261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb21fd-559d-4258-9b47-114efade2426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:53.275883 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb21fd-559d-4258-9b47-114efade2426 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:53.276158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb21fd-559d-4258-9b47-114efade2426 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:53.276425 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb21fd-559d-4258-9b47-114efade2426 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:53.283454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb21fd-559d-4258-9b47-114efade2426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:53.284917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb21fd-559d-4258-9b47-114efade2426 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-23bb21fd-559d-4258-9b47-114efade2426 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:55:53.295902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:55:53.297111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:55:53.297111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:55:53.297111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:55:53.297111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:55:53.297111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:55:53.297111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:55:53.297111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:55:53.297111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:55:53.297111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:55:53.297111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:55:53.297111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:55:53.297111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:55:53.297111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:55:53.297111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:55:53.297111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:55:53.297111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:55:53.298262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:55:53.298262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:55:53.298262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:55:53.298262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:55:53.298262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:55:53.298262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:55:53.298262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:55:53.298262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:55:53.298262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:55:53.298262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:55:53.298262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 06 07:55:53.298262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:55:53.298262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 06 07:55:53.298262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:55:53.298262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 575/1165] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Oct 6 07:55:53 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:55:53.303133 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067df47-d457-4b0f-ae36-6a3f96296e1e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:53.399836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067df47-d457-4b0f-ae36-6a3f96296e1e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:53.432502 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e48ae3-10f5-4a00-836e-2af1fc2f329d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:53.433099 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e48ae3-10f5-4a00-836e-2af1fc2f329d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:53.433370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e48ae3-10f5-4a00-836e-2af1fc2f329d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:53.433636 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e48ae3-10f5-4a00-836e-2af1fc2f329d None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:53.434264 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e48ae3-10f5-4a00-836e-2af1fc2f329d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:55:53.434557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e48ae3-10f5-4a00-836e-2af1fc2f329d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:53.435947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e48ae3-10f5-4a00-836e-2af1fc2f329d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:53.443577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 591/1166] 10.222.0.30 () {56 vars in 1115 bytes} [Thu Oct 6 07:55:53 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:55:55.547146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470cae7-4a64-4194-a64c-6d1d0ad09f09 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:55.547683 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470cae7-4a64-4194-a64c-6d1d0ad09f09 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:55.547683 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470cae7-4a64-4194-a64c-6d1d0ad09f09 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:55.549617 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 576/1167] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:55.554916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa99a59-b011-479f-935b-230e34b5f561 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:55.555372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa99a59-b011-479f-935b-230e34b5f561 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:55.555490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa99a59-b011-479f-935b-230e34b5f561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:55.573770 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7aa99a59-b011-479f-935b-230e34b5f561 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:55.574981 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7aa99a59-b011-479f-935b-230e34b5f561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:55.620936 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:55.642968 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 592/1168] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:55.910533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470cae7-4a64-4194-a64c-6d1d0ad09f09 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:56.007922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:56.017638 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470cae7-4a64-4194-a64c-6d1d0ad09f09 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:56.050859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff53764-698d-4bb6-b89e-5c3b61d82eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:56.051470 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff53764-698d-4bb6-b89e-5c3b61d82eb1 None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:56.051743 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff53764-698d-4bb6-b89e-5c3b61d82eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:56.052010 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff53764-698d-4bb6-b89e-5c3b61d82eb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:56.052667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff53764-698d-4bb6-b89e-5c3b61d82eb1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:56.054026 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff53764-698d-4bb6-b89e-5c3b61d82eb1 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:56.147017 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 577/1169] 10.222.0.30 () {60 vars in 1138 bytes} [Thu Oct 6 07:55:55 2022] POST /identity/v3/domains => generated 267 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:55:58.300291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb9d0f-7d72-43e5-a263-aec4750cc5ff None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:58.300806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb9d0f-7d72-43e5-a263-aec4750cc5ff None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:58.300806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb9d0f-7d72-43e5-a263-aec4750cc5ff None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:58.302158 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 593/1170] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:55:58.307584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951a100-c3fc-4011-bb68-528360e2527b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:58.307980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951a100-c3fc-4011-bb68-528360e2527b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:58.308141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951a100-c3fc-4011-bb68-528360e2527b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:58.326099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-e951a100-c3fc-4011-bb68-528360e2527b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:55:58.327323 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e951a100-c3fc-4011-bb68-528360e2527b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:55:58.392737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 578/1171] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:55:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:55:58.659339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb9d0f-7d72-43e5-a263-aec4750cc5ff None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:58.756641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb9d0f-7d72-43e5-a263-aec4750cc5ff None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:58.788742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d58182-3640-4094-b56a-49b681960c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:58.789345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d58182-3640-4094-b56a-49b681960c32 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:58.789617 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d58182-3640-4094-b56a-49b681960c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:58.789883 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d58182-3640-4094-b56a-49b681960c32 None admin] PATH_INFO: `/v3/domains/1fcc8974546840c29485543bf80d5fb8` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:58.799899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d58182-3640-4094-b56a-49b681960c32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1fcc8974546840c29485543bf80d5fb8)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:58.801313 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d58182-3640-4094-b56a-49b681960c32 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:58.802852 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 594/1172] 10.222.0.30 () {56 vars in 1183 bytes} [Thu Oct 6 07:55:58 2022] GET /identity/v3/domains/1fcc8974546840c29485543bf80d5fb8 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:55:58.808654 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951a100-c3fc-4011-bb68-528360e2527b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:55:58.886243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:55:58.903539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951a100-c3fc-4011-bb68-528360e2527b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:55:58.937154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944ac59-22b5-409d-9c63-9724f05e1668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:55:58.937678 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944ac59-22b5-409d-9c63-9724f05e1668 None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:55:58.937952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944ac59-22b5-409d-9c63-9724f05e1668 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:55:58.939266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944ac59-22b5-409d-9c63-9724f05e1668 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:55:58.939571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944ac59-22b5-409d-9c63-9724f05e1668 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:55:58.940988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944ac59-22b5-409d-9c63-9724f05e1668 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:55:59.106284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 579/1173] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Oct 6 07:55:58 2022] POST /identity/v3/projects => generated 361 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:56:01.309213 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bcf89-55a7-4cab-9f88-aadf02027ce3 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:01.309734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bcf89-55a7-4cab-9f88-aadf02027ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:01.309734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bcf89-55a7-4cab-9f88-aadf02027ce3 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:01.311304 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 595/1174] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:56:01.317929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cadbf-e782-4376-a060-9dc4d2ca0beb None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:01.318372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cadbf-e782-4376-a060-9dc4d2ca0beb None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:01.318483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cadbf-e782-4376-a060-9dc4d2ca0beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:01.336757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-db9cadbf-e782-4376-a060-9dc4d2ca0beb None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:56:01.337925 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-db9cadbf-e782-4376-a060-9dc4d2ca0beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:56:01.406893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 580/1175] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:56:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:56:01.681823 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bcf89-55a7-4cab-9f88-aadf02027ce3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:01.724816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:01.781447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bcf89-55a7-4cab-9f88-aadf02027ce3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:01.814710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef22898-e591-4707-a24a-0f653ab06b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:01.815362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef22898-e591-4707-a24a-0f653ab06b45 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:01.815637 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef22898-e591-4707-a24a-0f653ab06b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:01.815921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef22898-e591-4707-a24a-0f653ab06b45 None admin] PATH_INFO: `/v3/domains/1fcc8974546840c29485543bf80d5fb8` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:01.817015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef22898-e591-4707-a24a-0f653ab06b45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1fcc8974546840c29485543bf80d5fb8)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:56:01.818435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef22898-e591-4707-a24a-0f653ab06b45 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:56:01.820041 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 596/1176] 10.222.0.30 () {56 vars in 1183 bytes} [Thu Oct 6 07:56:01 2022] GET /identity/v3/domains/1fcc8974546840c29485543bf80d5fb8 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:56:01.825779 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cadbf-e782-4376-a060-9dc4d2ca0beb None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:01.886205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:01.921974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cadbf-e782-4376-a060-9dc4d2ca0beb None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:01.968889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010276c-d425-4003-8c20-8cec8cec0019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:01.968889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010276c-d425-4003-8c20-8cec8cec0019 None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:01.968889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010276c-d425-4003-8c20-8cec8cec0019 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:01.968889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010276c-d425-4003-8c20-8cec8cec0019 None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:01.969835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010276c-d425-4003-8c20-8cec8cec0019 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:56:01.969835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010276c-d425-4003-8c20-8cec8cec0019 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:56:02.070738 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 581/1177] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:56:01 2022] POST /identity/v3/users => generated 311 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:56:04.314958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025c8cb-87b5-4211-b410-42e17cbf46c8 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:04.315527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025c8cb-87b5-4211-b410-42e17cbf46c8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:04.315527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025c8cb-87b5-4211-b410-42e17cbf46c8 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:04.316867 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 597/1178] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:56:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:56:04.322124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a039a8-a89d-4fee-9190-68b20153221b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:04.322519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a039a8-a89d-4fee-9190-68b20153221b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:04.322666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a039a8-a89d-4fee-9190-68b20153221b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:04.340561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-f3a039a8-a89d-4fee-9190-68b20153221b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:56:04.341745 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f3a039a8-a89d-4fee-9190-68b20153221b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:56:04.407401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 582/1179] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:56:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:56:04.675993 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025c8cb-87b5-4211-b410-42e17cbf46c8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:04.774808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025c8cb-87b5-4211-b410-42e17cbf46c8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:04.785246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:04.807604 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd91413-51db-4871-a87a-de340e99e90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:04.808213 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd91413-51db-4871-a87a-de340e99e90e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:04.808486 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd91413-51db-4871-a87a-de340e99e90e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:04.808755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd91413-51db-4871-a87a-de340e99e90e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:04.819589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd91413-51db-4871-a87a-de340e99e90e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:56:04.824277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1dd91413-51db-4871-a87a-de340e99e90e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:56:04.824277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:56:04.824277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:56:04.824277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:56:04.824277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:56:04.824277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:56:04.824277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:56:04.824277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:56:04.824277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:56:04.824277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:56:04.824277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:56:04.824277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:56:04.824277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:56:04.824277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:56:04.824277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:56:04.824277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:56:04.825365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:56:04.825365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:56:04.825365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:56:04.825365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:56:04.825365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:56:04.825365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:56:04.825365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:56:04.825365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:56:04.825365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:56:04.825365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:56:04.825365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:56:04.825365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:56:04.825365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:56:04.825365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:56:04.825365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:56:04.825365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:56:04.826395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:56:04.826395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:56:04.826395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:56:04.826395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:56:04.826395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:56:04.826395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:56:04.826395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:56:04.826395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:56:04.826395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:56:04.826395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:56:04.826395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:56:04.826395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:56:04.826395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 598/1180] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:56:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:56:04.831986 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a039a8-a89d-4fee-9190-68b20153221b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:04.880159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:04.928759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a039a8-a89d-4fee-9190-68b20153221b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:04.962493 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c55b4-2b27-4141-9198-c9eb801a0bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:04.963115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c55b4-2b27-4141-9198-c9eb801a0bac None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:04.963396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c55b4-2b27-4141-9198-c9eb801a0bac None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:04.963669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c55b4-2b27-4141-9198-c9eb801a0bac None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:04.964327 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c55b4-2b27-4141-9198-c9eb801a0bac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:56:04.964626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c55b4-2b27-4141-9198-c9eb801a0bac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:56:04.966005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c55b4-2b27-4141-9198-c9eb801a0bac None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:56:04.975114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 583/1181] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:56:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:56:04.981457 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd91413-51db-4871-a87a-de340e99e90e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:05.084480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd91413-51db-4871-a87a-de340e99e90e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:05.119609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c56e1-8fbe-40f1-95df-82b8cbaf789c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:05.120227 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c56e1-8fbe-40f1-95df-82b8cbaf789c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:05.120498 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c56e1-8fbe-40f1-95df-82b8cbaf789c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:05.120759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c56e1-8fbe-40f1-95df-82b8cbaf789c None admin] PATH_INFO: `/v3/users/7d53fa3cd566411a8231851723cbaf16` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:05.134914 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c56e1-8fbe-40f1-95df-82b8cbaf789c None admin] RBAC: Authorizing `identity:get_user(user_id=7d53fa3cd566411a8231851723cbaf16)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:56:05.136377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c56e1-8fbe-40f1-95df-82b8cbaf789c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:56:05.137968 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 599/1182] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:56:04 2022] GET /identity/v3/users/7d53fa3cd566411a8231851723cbaf16 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:56:05.143869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c55b4-2b27-4141-9198-c9eb801a0bac None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:05.239925 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c55b4-2b27-4141-9198-c9eb801a0bac None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:05.253006 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:05.273002 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079508d-7e6a-47eb-b1ed-693c5552d346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:05.273597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079508d-7e6a-47eb-b1ed-693c5552d346 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:05.273875 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079508d-7e6a-47eb-b1ed-693c5552d346 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:05.274147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079508d-7e6a-47eb-b1ed-693c5552d346 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:05.344587 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079508d-7e6a-47eb-b1ed-693c5552d346 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:56:05.346203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079508d-7e6a-47eb-b1ed-693c5552d346 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:56:05.441923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 584/1183] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:56:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 07:56:05.447919 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c56e1-8fbe-40f1-95df-82b8cbaf789c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:05.475393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:05.544951 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c56e1-8fbe-40f1-95df-82b8cbaf789c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:05.577526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df9daf-e184-421f-943c-a42edf729e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:05.578138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df9daf-e184-421f-943c-a42edf729e3e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:05.578416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df9daf-e184-421f-943c-a42edf729e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:05.578801 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df9daf-e184-421f-943c-a42edf729e3e None admin] PATH_INFO: `/v3/projects/8bbb4c852e6a4b65be5b5cc3eb88040e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:05.580029 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df9daf-e184-421f-943c-a42edf729e3e None admin] RBAC: Authorizing `identity:get_project(project_id=8bbb4c852e6a4b65be5b5cc3eb88040e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:56:05.581450 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df9daf-e184-421f-943c-a42edf729e3e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:56:05.583082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 600/1184] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:56:05 2022] GET /identity/v3/projects/8bbb4c852e6a4b65be5b5cc3eb88040e => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:56:05.589076 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079508d-7e6a-47eb-b1ed-693c5552d346 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:05.603300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:05.689617 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079508d-7e6a-47eb-b1ed-693c5552d346 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:05.722435 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f38fd69-79e5-42ca-b3fb-4dddaa31a9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:05.723052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38fd69-79e5-42ca-b3fb-4dddaa31a9aa None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:05.723332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38fd69-79e5-42ca-b3fb-4dddaa31a9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:05.723603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38fd69-79e5-42ca-b3fb-4dddaa31a9aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:05.724324 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f38fd69-79e5-42ca-b3fb-4dddaa31a9aa None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=8bbb4c852e6a4b65be5b5cc3eb88040e, user.id=7d53fa3cd566411a8231851723cbaf16) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:56:05.724629 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f38fd69-79e5-42ca-b3fb-4dddaa31a9aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:56:05.726097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f38fd69-79e5-42ca-b3fb-4dddaa31a9aa None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:56:05.731341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 585/1185] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:56:05 2022] GET /identity/v3/role_assignments?user.id=7d53fa3cd566411a8231851723cbaf16&scope.project.id=8bbb4c852e6a4b65be5b5cc3eb88040e&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:56:07.845353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d52c7b-cf3c-46c0-b661-1400103d8399 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:07.845882 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d52c7b-cf3c-46c0-b661-1400103d8399 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:07.845882 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d52c7b-cf3c-46c0-b661-1400103d8399 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:07.847259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 601/1186] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:56:07.852403 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c687512-d7dd-4dc7-82c8-8d89dc1e6a5a None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:07.852833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c687512-d7dd-4dc7-82c8-8d89dc1e6a5a None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:07.852949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c687512-d7dd-4dc7-82c8-8d89dc1e6a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:07.870901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-9c687512-d7dd-4dc7-82c8-8d89dc1e6a5a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:56:07.872118 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9c687512-d7dd-4dc7-82c8-8d89dc1e6a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:56:07.938357 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 586/1187] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:56:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:56:08.211073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d52c7b-cf3c-46c0-b661-1400103d8399 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:08.302065 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:08.308291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d52c7b-cf3c-46c0-b661-1400103d8399 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:08.340350 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc2c1f-faa3-4207-b645-774cbbc13325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:08.340951 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc2c1f-faa3-4207-b645-774cbbc13325 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:08.341225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc2c1f-faa3-4207-b645-774cbbc13325 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:08.341489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc2c1f-faa3-4207-b645-774cbbc13325 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:08.351673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfc2c1f-faa3-4207-b645-774cbbc13325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:56:08.356319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-bcfc2c1f-faa3-4207-b645-774cbbc13325 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:56:08.356319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:56:08.356319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:56:08.356319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:56:08.356319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:56:08.356319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:56:08.356319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:56:08.356319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:56:08.356319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:56:08.356319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:56:08.356319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:56:08.356319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:56:08.356319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:56:08.356319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:56:08.356319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:56:08.356319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:56:08.357384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:56:08.357384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:56:08.357384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:56:08.357384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:56:08.357384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:56:08.357384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:56:08.357384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:56:08.357384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:56:08.357384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:56:08.357384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:56:08.357384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:56:08.357384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:56:08.357384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:56:08.357384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:56:08.357384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:56:08.357384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:56:08.358252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:56:08.358252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:56:08.358252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:56:08.358252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:56:08.358252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:56:08.358252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:56:08.358252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:56:08.358252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:56:08.358252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:56:08.358252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:56:08.358252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:56:08.358252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:56:08.358252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 602/1188] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:56:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:56:08.363517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c687512-d7dd-4dc7-82c8-8d89dc1e6a5a None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:08.371408 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:08.458807 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c687512-d7dd-4dc7-82c8-8d89dc1e6a5a None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:08.490409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b7142-710a-4bb9-9914-92f10bb2da22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:08.491014 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b7142-710a-4bb9-9914-92f10bb2da22 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:08.491288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b7142-710a-4bb9-9914-92f10bb2da22 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:08.491553 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b7142-710a-4bb9-9914-92f10bb2da22 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:08.492200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b7142-710a-4bb9-9914-92f10bb2da22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:56:08.492499 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b7142-710a-4bb9-9914-92f10bb2da22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:56:08.493858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b7142-710a-4bb9-9914-92f10bb2da22 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:56:08.502308 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 587/1189] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:56:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:56:08.508115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc2c1f-faa3-4207-b645-774cbbc13325 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:08.605276 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc2c1f-faa3-4207-b645-774cbbc13325 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:08.638388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354aec8d-0421-4d5a-82b3-f2112b686d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:08.639025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aec8d-0421-4d5a-82b3-f2112b686d77 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:08.639304 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aec8d-0421-4d5a-82b3-f2112b686d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:08.639576 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aec8d-0421-4d5a-82b3-f2112b686d77 None admin] PATH_INFO: `/v3/users/7d53fa3cd566411a8231851723cbaf16` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:08.640685 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354aec8d-0421-4d5a-82b3-f2112b686d77 None admin] RBAC: Authorizing `identity:get_user(user_id=7d53fa3cd566411a8231851723cbaf16)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:56:08.642121 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354aec8d-0421-4d5a-82b3-f2112b686d77 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:56:08.643705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 603/1190] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:56:08 2022] GET /identity/v3/users/7d53fa3cd566411a8231851723cbaf16 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:56:08.649538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b7142-710a-4bb9-9914-92f10bb2da22 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:08.714331 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:08.746751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b7142-710a-4bb9-9914-92f10bb2da22 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:08.778877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549c3f8-4250-443c-a388-ee84067a3f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:08.779525 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549c3f8-4250-443c-a388-ee84067a3f89 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:08.779797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549c3f8-4250-443c-a388-ee84067a3f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:08.780064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549c3f8-4250-443c-a388-ee84067a3f89 None admin] PATH_INFO: `/v3/projects/8bbb4c852e6a4b65be5b5cc3eb88040e` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:08.781148 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549c3f8-4250-443c-a388-ee84067a3f89 None admin] RBAC: Authorizing `identity:get_project(project_id=8bbb4c852e6a4b65be5b5cc3eb88040e)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:56:08.783198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549c3f8-4250-443c-a388-ee84067a3f89 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:56:08.784796 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 588/1191] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:56:08 2022] GET /identity/v3/projects/8bbb4c852e6a4b65be5b5cc3eb88040e => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:56:08.790764 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354aec8d-0421-4d5a-82b3-f2112b686d77 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:08.816247 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:08.888436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354aec8d-0421-4d5a-82b3-f2112b686d77 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:08.921304 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed1a86-6cc2-44a5-aeb8-55d64dd9abe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:08.921934 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed1a86-6cc2-44a5-aeb8-55d64dd9abe8 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:08.922233 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed1a86-6cc2-44a5-aeb8-55d64dd9abe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:08.922500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed1a86-6cc2-44a5-aeb8-55d64dd9abe8 None admin] PATH_INFO: `/v3/projects/8bbb4c852e6a4b65be5b5cc3eb88040e/users/7d53fa3cd566411a8231851723cbaf16/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:08.924366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ed1a86-6cc2-44a5-aeb8-55d64dd9abe8 None admin] RBAC: Authorizing `identity:create_grant(project_id=8bbb4c852e6a4b65be5b5cc3eb88040e, user_id=7d53fa3cd566411a8231851723cbaf16, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:56:08.925763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ed1a86-6cc2-44a5-aeb8-55d64dd9abe8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:56:08.990415 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 604/1192] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:56:08 2022] PUT /identity/v3/projects/8bbb4c852e6a4b65be5b5cc3eb88040e/users/7d53fa3cd566411a8231851723cbaf16/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:56:11.104132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76a963-26d7-499c-889f-aea548767a58 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:11.104695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76a963-26d7-499c-889f-aea548767a58 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:11.104695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76a963-26d7-499c-889f-aea548767a58 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:11.106050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 589/1193] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:56:11.111937 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de4e8c-811d-40a3-b635-1511ee7645da None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:11.112321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de4e8c-811d-40a3-b635-1511ee7645da None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:11.112475 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de4e8c-811d-40a3-b635-1511ee7645da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:11.131650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-90de4e8c-811d-40a3-b635-1511ee7645da None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:56:11.132878 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-90de4e8c-811d-40a3-b635-1511ee7645da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:56:11.201755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 605/1194] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:56:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:56:11.474517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76a963-26d7-499c-889f-aea548767a58 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:11.572846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76a963-26d7-499c-889f-aea548767a58 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:11.605504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed69028-d2ab-4a60-bfc7-7801425f1d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:11.606120 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed69028-d2ab-4a60-bfc7-7801425f1d0a None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:11.606399 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed69028-d2ab-4a60-bfc7-7801425f1d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:11.606674 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed69028-d2ab-4a60-bfc7-7801425f1d0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:11.609224 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:11.618760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed69028-d2ab-4a60-bfc7-7801425f1d0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:56:11.623760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-4ed69028-d2ab-4a60-bfc7-7801425f1d0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:56:11.623760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:56:11.623760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:56:11.623760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:56:11.623760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:56:11.623760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:56:11.623760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:56:11.623760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:56:11.623760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:56:11.623760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:56:11.623760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:56:11.623760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:56:11.623760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:56:11.623760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:56:11.623760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:56:11.623760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:56:11.624934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:56:11.624934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:56:11.624934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:56:11.624934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:56:11.624934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:56:11.624934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:56:11.624934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:56:11.624934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:56:11.624934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:56:11.624934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:56:11.624934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:56:11.624934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:56:11.624934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:56:11.624934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:56:11.624934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:56:11.624934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:56:11.626300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:56:11.626300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:56:11.626300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:56:11.626300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:56:11.626300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:56:11.626300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:56:11.626300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:56:11.626300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:56:11.626300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:56:11.626300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:56:11.626300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:56:11.626300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:56:11.626300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 590/1195] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:56:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:56:11.631411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de4e8c-811d-40a3-b635-1511ee7645da None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:11.647358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:11.732478 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de4e8c-811d-40a3-b635-1511ee7645da None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:11.765225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270459ac-3ea9-4375-9f5b-34cb99b5538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:11.766748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270459ac-3ea9-4375-9f5b-34cb99b5538f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:11.767054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270459ac-3ea9-4375-9f5b-34cb99b5538f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:11.767325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270459ac-3ea9-4375-9f5b-34cb99b5538f None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:11.767984 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270459ac-3ea9-4375-9f5b-34cb99b5538f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:56:11.768284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270459ac-3ea9-4375-9f5b-34cb99b5538f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:56:11.769634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270459ac-3ea9-4375-9f5b-34cb99b5538f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:56:11.777307 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 606/1196] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:56:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:56:11.784422 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed69028-d2ab-4a60-bfc7-7801425f1d0a None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:11.881581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed69028-d2ab-4a60-bfc7-7801425f1d0a None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:11.914750 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa185d2-9b18-48cb-b019-1b05a328c392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:11.915389 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa185d2-9b18-48cb-b019-1b05a328c392 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:11.915664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa185d2-9b18-48cb-b019-1b05a328c392 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:11.915932 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa185d2-9b18-48cb-b019-1b05a328c392 None admin] PATH_INFO: `/v3/users/7d53fa3cd566411a8231851723cbaf16` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:11.917032 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa185d2-9b18-48cb-b019-1b05a328c392 None admin] RBAC: Authorizing `identity:get_user(user_id=7d53fa3cd566411a8231851723cbaf16)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:56:11.918457 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa185d2-9b18-48cb-b019-1b05a328c392 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:56:11.920030 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 591/1197] 10.222.0.30 () {56 vars in 1177 bytes} [Thu Oct 6 07:56:11 2022] GET /identity/v3/users/7d53fa3cd566411a8231851723cbaf16 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:56:11.925727 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270459ac-3ea9-4375-9f5b-34cb99b5538f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:12.000748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:12.022718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270459ac-3ea9-4375-9f5b-34cb99b5538f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:12.055588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a47a5-e467-428f-9aec-fbea4b800772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:12.056183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a47a5-e467-428f-9aec-fbea4b800772 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:12.056458 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a47a5-e467-428f-9aec-fbea4b800772 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:12.056725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a47a5-e467-428f-9aec-fbea4b800772 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:12.120685 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a47a5-e467-428f-9aec-fbea4b800772 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:56:12.122113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a47a5-e467-428f-9aec-fbea4b800772 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:56:12.194400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:12.216397 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 607/1198] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:56:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 07:56:12.222899 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa185d2-9b18-48cb-b019-1b05a328c392 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:12.282764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:12.318846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa185d2-9b18-48cb-b019-1b05a328c392 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:12.350949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010b731-c372-42db-aac7-70d6c628fbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:12.351568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010b731-c372-42db-aac7-70d6c628fbc5 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:12.351841 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010b731-c372-42db-aac7-70d6c628fbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:12.352106 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010b731-c372-42db-aac7-70d6c628fbc5 None admin] PATH_INFO: `/v3/projects/8bbb4c852e6a4b65be5b5cc3eb88040e` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:12.353196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010b731-c372-42db-aac7-70d6c628fbc5 None admin] RBAC: Authorizing `identity:get_project(project_id=8bbb4c852e6a4b65be5b5cc3eb88040e)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:56:12.354616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010b731-c372-42db-aac7-70d6c628fbc5 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:56:12.356225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 592/1199] 10.222.0.30 () {56 vars in 1186 bytes} [Thu Oct 6 07:56:12 2022] GET /identity/v3/projects/8bbb4c852e6a4b65be5b5cc3eb88040e => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:56:12.362795 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a47a5-e467-428f-9aec-fbea4b800772 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:56:12.456484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a47a5-e467-428f-9aec-fbea4b800772 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:56:12.489221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fc7b1-5a6f-48fe-8229-756001af0871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:56:12.489798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fc7b1-5a6f-48fe-8229-756001af0871 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:56:12.490067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fc7b1-5a6f-48fe-8229-756001af0871 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:56:12.490331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fc7b1-5a6f-48fe-8229-756001af0871 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:56:12.491064 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fc7b1-5a6f-48fe-8229-756001af0871 None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=8bbb4c852e6a4b65be5b5cc3eb88040e, user.id=7d53fa3cd566411a8231851723cbaf16) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:56:12.491370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fc7b1-5a6f-48fe-8229-756001af0871 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:56:12.492767 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fc7b1-5a6f-48fe-8229-756001af0871 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:56:12.498476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 608/1200] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:56:12 2022] GET /identity/v3/role_assignments?user.id=7d53fa3cd566411a8231851723cbaf16&scope.project.id=8bbb4c852e6a4b65be5b5cc3eb88040e&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:56:22.346956 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=34 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:56:22.495761 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=80 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:25.695525 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb615c-d523-4d7a-bade-42995b4e3b3f None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:25.696330 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb615c-d523-4d7a-bade-42995b4e3b3f None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:25.696330 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb615c-d523-4d7a-bade-42995b4e3b3f None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:25.697413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 593/1201] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:57:25.702784 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30ed24-c3c1-42c0-9da0-d7bf78c58982 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:25.703221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30ed24-c3c1-42c0-9da0-d7bf78c58982 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:25.703353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30ed24-c3c1-42c0-9da0-d7bf78c58982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:25.722071 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5e30ed24-c3c1-42c0-9da0-d7bf78c58982 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:57:25.726264 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5e30ed24-c3c1-42c0-9da0-d7bf78c58982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:57:25.789774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 609/1202] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:57:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:57:26.062213 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb615c-d523-4d7a-bade-42995b4e3b3f None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:26.164748 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb615c-d523-4d7a-bade-42995b4e3b3f None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:26.198678 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99d71e-fe78-4340-8fd0-4e337f6a24b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:26.199329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99d71e-fe78-4340-8fd0-4e337f6a24b7 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:26.199606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99d71e-fe78-4340-8fd0-4e337f6a24b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:26.199876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99d71e-fe78-4340-8fd0-4e337f6a24b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:26.210366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99d71e-fe78-4340-8fd0-4e337f6a24b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:26.211843 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99d71e-fe78-4340-8fd0-4e337f6a24b7 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:26.226306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-2a99d71e-fe78-4340-8fd0-4e337f6a24b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:26.226306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:26.226306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:26.226306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:26.226306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:26.226306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:26.226306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:26.226306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:26.226306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:26.226306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:26.226306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:26.226306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:26.226306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:57:26.226306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:57:26.226306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:57:26.226306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:26.227443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:57:26.228809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:57:26.228809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:26.228809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:57:26.228809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 594/1203] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:26.233581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30ed24-c3c1-42c0-9da0-d7bf78c58982 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:26.333991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30ed24-c3c1-42c0-9da0-d7bf78c58982 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:26.363174 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:26.368841 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c16eb-1ee2-4f69-8e5e-08ef38759f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:26.370109 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c16eb-1ee2-4f69-8e5e-08ef38759f7a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:26.370109 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c16eb-1ee2-4f69-8e5e-08ef38759f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:26.370109 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c16eb-1ee2-4f69-8e5e-08ef38759f7a None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:26.370731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c16eb-1ee2-4f69-8e5e-08ef38759f7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:26.371074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c16eb-1ee2-4f69-8e5e-08ef38759f7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:26.372688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c16eb-1ee2-4f69-8e5e-08ef38759f7a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:26.384091 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 610/1204] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:26.389798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99d71e-fe78-4340-8fd0-4e337f6a24b7 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:26.442611 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:26.487911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99d71e-fe78-4340-8fd0-4e337f6a24b7 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:26.520920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e2bca-8f17-46e8-b914-cbebc419cbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:26.521508 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e2bca-8f17-46e8-b914-cbebc419cbab None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:26.521779 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e2bca-8f17-46e8-b914-cbebc419cbab None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:26.522044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e2bca-8f17-46e8-b914-cbebc419cbab None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:26.522666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e2bca-8f17-46e8-b914-cbebc419cbab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:26.524118 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e2bca-8f17-46e8-b914-cbebc419cbab None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:26.611261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 595/1205] 10.222.0.30 () {60 vars in 1132 bytes} [Thu Oct 6 07:57:26 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:57:28.803929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6110555-1ec2-4d7c-a475-3f985c007830 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:28.804479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6110555-1ec2-4d7c-a475-3f985c007830 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:28.804479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6110555-1ec2-4d7c-a475-3f985c007830 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:28.805819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 611/1206] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:57:28.811074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f72239-9d6f-4d4a-9412-fa45502e0835 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:28.811458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f72239-9d6f-4d4a-9412-fa45502e0835 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:28.811643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f72239-9d6f-4d4a-9412-fa45502e0835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:28.831051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-89f72239-9d6f-4d4a-9412-fa45502e0835 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:57:28.832298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-89f72239-9d6f-4d4a-9412-fa45502e0835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:57:28.898310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 596/1207] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:57:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:57:29.168739 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6110555-1ec2-4d7c-a475-3f985c007830 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:29.271054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6110555-1ec2-4d7c-a475-3f985c007830 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:29.304977 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9530d8b-20d0-4e69-a078-c4a42883654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:29.305599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9530d8b-20d0-4e69-a078-c4a42883654c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:29.305874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9530d8b-20d0-4e69-a078-c4a42883654c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:29.306144 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9530d8b-20d0-4e69-a078-c4a42883654c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:29.316236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9530d8b-20d0-4e69-a078-c4a42883654c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:57:29.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-b9530d8b-20d0-4e69-a078-c4a42883654c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:57:29.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:29.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:29.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:29.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:29.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:29.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:29.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:29.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:29.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:29.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:29.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:29.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:57:29.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:57:29.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:57:29.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:57:29.329579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:57:29.329579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:57:29.329579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:57:29.329579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:57:29.329579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:29.329579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:29.329579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:29.329579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:29.329579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:29.329579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:29.329579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:29.329579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:29.329579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:29.329579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:29.329579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:29.329579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:29.330755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:29.330755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:29.330755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:29.330755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:29.330755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:57:29.330755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:57:29.330755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:57:29.330755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:57:29.330755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:57:29.330755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:57:29.330755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:57:29.330755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:29.330755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 612/1208] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:57:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:29.331702 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f72239-9d6f-4d4a-9412-fa45502e0835 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:29.368161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:29.433130 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f72239-9d6f-4d4a-9412-fa45502e0835 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:29.465676 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86430cf3-0cfd-47f1-96e9-6b2173a82088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:29.466270 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86430cf3-0cfd-47f1-96e9-6b2173a82088 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:29.466592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86430cf3-0cfd-47f1-96e9-6b2173a82088 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:29.466858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86430cf3-0cfd-47f1-96e9-6b2173a82088 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:29.467547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86430cf3-0cfd-47f1-96e9-6b2173a82088 None admin] RBAC: Adding query filter params (name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:29.467848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86430cf3-0cfd-47f1-96e9-6b2173a82088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:29.469213 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86430cf3-0cfd-47f1-96e9-6b2173a82088 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:29.477507 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 597/1209] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:57:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:29.483418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9530d8b-20d0-4e69-a078-c4a42883654c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:29.533783 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:29.582588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9530d8b-20d0-4e69-a078-c4a42883654c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:29.614804 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c94ba3-b414-4562-b926-0edb4ba53504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:29.615592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c94ba3-b414-4562-b926-0edb4ba53504 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:29.615863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c94ba3-b414-4562-b926-0edb4ba53504 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:29.616128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c94ba3-b414-4562-b926-0edb4ba53504 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:29.626266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c94ba3-b414-4562-b926-0edb4ba53504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:29.627882 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c94ba3-b414-4562-b926-0edb4ba53504 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:29.641755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-15c94ba3-b414-4562-b926-0edb4ba53504 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:29.641755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:29.641755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:29.641755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:29.641755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:29.641755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:29.641755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:29.641755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:29.641755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:29.641755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:29.641755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:29.641755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:29.641755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:57:29.641755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:57:29.641755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:57:29.641755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:29.643881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:57:29.644850 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:57:29.644850 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:29.644850 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:29.644850 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 613/1210] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:29.649396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86430cf3-0cfd-47f1-96e9-6b2173a82088 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:29.746792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:29.749988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86430cf3-0cfd-47f1-96e9-6b2173a82088 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:29.785240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cecef-f9d0-48e6-bbdf-33da259a1f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:29.785835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cecef-f9d0-48e6-bbdf-33da259a1f66 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:29.786109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cecef-f9d0-48e6-bbdf-33da259a1f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:29.786377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cecef-f9d0-48e6-bbdf-33da259a1f66 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:29.787035 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cecef-f9d0-48e6-bbdf-33da259a1f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:29.787338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cecef-f9d0-48e6-bbdf-33da259a1f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:29.788701 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cecef-f9d0-48e6-bbdf-33da259a1f66 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:29.801120 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 598/1211] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:29.806799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c94ba3-b414-4562-b926-0edb4ba53504 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:29.907359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c94ba3-b414-4562-b926-0edb4ba53504 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:29.909128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:29.942523 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28171a87-0924-4932-8a1a-7cc1165992ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:29.943161 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28171a87-0924-4932-8a1a-7cc1165992ea None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:29.943433 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28171a87-0924-4932-8a1a-7cc1165992ea None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:29.943697 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28171a87-0924-4932-8a1a-7cc1165992ea None admin] PATH_INFO: `/v3/users/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:29.950382 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28171a87-0924-4932-8a1a-7cc1165992ea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:29.951867 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28171a87-0924-4932-8a1a-7cc1165992ea None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:29.963346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-28171a87-0924-4932-8a1a-7cc1165992ea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 07:57:29.963346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:29.963346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:29.963346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:29.963346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:29.963346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:29.963346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:29.963346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:29.963346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:29.963346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:29.963346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:29.963346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:29.963346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:57:29.963346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:57:29.963346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:57:29.963346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:29.964344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:29.965355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:29.965355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:29.965355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:57:29.965355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:57:29.965355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:57:29.965355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:57:29.965355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:57:29.965355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:57:29.965355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 07:57:29.965355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:29.965355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 614/1212] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:29.970399 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cecef-f9d0-48e6-bbdf-33da259a1f66 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:30.070296 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cecef-f9d0-48e6-bbdf-33da259a1f66 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:30.103804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daae5ce-71c1-4be4-8ec7-b301d58382e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:30.104398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daae5ce-71c1-4be4-8ec7-b301d58382e7 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:30.104673 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daae5ce-71c1-4be4-8ec7-b301d58382e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:30.104939 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daae5ce-71c1-4be4-8ec7-b301d58382e7 None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:30.105657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daae5ce-71c1-4be4-8ec7-b301d58382e7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:30.105956 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daae5ce-71c1-4be4-8ec7-b301d58382e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:30.107426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daae5ce-71c1-4be4-8ec7-b301d58382e7 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:30.125255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 599/1213] 10.222.0.30 () {56 vars in 1143 bytes} [Thu Oct 6 07:57:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:30.131680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28171a87-0924-4932-8a1a-7cc1165992ea None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:30.235826 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28171a87-0924-4932-8a1a-7cc1165992ea None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:30.269266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ad4a2-f202-4ccc-a498-caa6a4d65173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:30.269855 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ad4a2-f202-4ccc-a498-caa6a4d65173 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:30.270125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ad4a2-f202-4ccc-a498-caa6a4d65173 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:30.270387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ad4a2-f202-4ccc-a498-caa6a4d65173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:30.302757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:30.340105 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ad4a2-f202-4ccc-a498-caa6a4d65173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:30.341538 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ad4a2-f202-4ccc-a498-caa6a4d65173 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:30.440681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 615/1214] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:57:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 07:57:30.446683 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daae5ce-71c1-4be4-8ec7-b301d58382e7 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:30.465680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:30.542288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daae5ce-71c1-4be4-8ec7-b301d58382e7 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:30.575134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67f94e-84c1-4346-b80f-898b10fa3da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:30.575749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67f94e-84c1-4346-b80f-898b10fa3da7 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:30.576021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67f94e-84c1-4346-b80f-898b10fa3da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:30.576287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67f94e-84c1-4346-b80f-898b10fa3da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:30.586312 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67f94e-84c1-4346-b80f-898b10fa3da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:30.587777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67f94e-84c1-4346-b80f-898b10fa3da7 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:30.602757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-1b67f94e-84c1-4346-b80f-898b10fa3da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:30.602757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:30.602757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:30.602757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:30.602757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:30.602757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:30.602757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:30.602757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:30.602757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:30.602757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:30.602757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:30.602757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:30.602757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:57:30.602757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:57:30.602757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:57:30.602757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:30.603949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:57:30.604907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:57:30.604907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:30.604907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:57:30.604907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 600/1215] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:30.609594 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ad4a2-f202-4ccc-a498-caa6a4d65173 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:30.670119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:30.709172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ad4a2-f202-4ccc-a498-caa6a4d65173 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:30.740668 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da738ca-51a9-439c-b346-d296c8501a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:30.741263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da738ca-51a9-439c-b346-d296c8501a30 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:30.741540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da738ca-51a9-439c-b346-d296c8501a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:30.741814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da738ca-51a9-439c-b346-d296c8501a30 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:30.742453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da738ca-51a9-439c-b346-d296c8501a30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:30.742753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da738ca-51a9-439c-b346-d296c8501a30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:30.744148 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da738ca-51a9-439c-b346-d296c8501a30 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:30.756394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 616/1216] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:30.762027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67f94e-84c1-4346-b80f-898b10fa3da7 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:30.829778 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:30.859538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67f94e-84c1-4346-b80f-898b10fa3da7 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:30.892414 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a5446-cd1e-4305-bf11-57c0393a1a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:30.893898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a5446-cd1e-4305-bf11-57c0393a1a31 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:30.894176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a5446-cd1e-4305-bf11-57c0393a1a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:30.894471 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a5446-cd1e-4305-bf11-57c0393a1a31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:30.901476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a5446-cd1e-4305-bf11-57c0393a1a31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:30.902999 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a5446-cd1e-4305-bf11-57c0393a1a31 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:30.913846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-f99a5446-cd1e-4305-bf11-57c0393a1a31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:57:30.913846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:30.913846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:30.913846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:30.913846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:30.913846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:30.913846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:30.913846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:30.913846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:30.913846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:30.913846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:30.913846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:30.913846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:57:30.913846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:57:30.913846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:30.914893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:30.915877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:57:30.915877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:57:30.915877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:57:30.915877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:57:30.915877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:57:30.915877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:57:30.915877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:57:30.915877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:57:30.915877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 601/1217] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:57:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:30.920989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da738ca-51a9-439c-b346-d296c8501a30 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:31.019852 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da738ca-51a9-439c-b346-d296c8501a30 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:31.042265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:31.053810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e78415-8959-4740-a623-6d913525d1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:31.054424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e78415-8959-4740-a623-6d913525d1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:31.054674 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e78415-8959-4740-a623-6d913525d1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:31.054988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e78415-8959-4740-a623-6d913525d1d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:31.055713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e78415-8959-4740-a623-6d913525d1d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:31.056005 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e78415-8959-4740-a623-6d913525d1d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:31.057416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e78415-8959-4740-a623-6d913525d1d9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:31.068918 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 617/1218] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:57:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:31.075191 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a5446-cd1e-4305-bf11-57c0393a1a31 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:31.176321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a5446-cd1e-4305-bf11-57c0393a1a31 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:31.205485 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:31.212328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623f69e-1000-4a74-81d6-75a82fe9f198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:31.212328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623f69e-1000-4a74-81d6-75a82fe9f198 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:31.212328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623f69e-1000-4a74-81d6-75a82fe9f198 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:31.212328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623f69e-1000-4a74-81d6-75a82fe9f198 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:31.213057 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623f69e-1000-4a74-81d6-75a82fe9f198 None admin] RBAC: Adding query filter params (role.id=9f0e36c2ed8843fbabd3a4cf30bc3725, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=6074872303d3468fb566378fa34e8bea) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:31.213366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623f69e-1000-4a74-81d6-75a82fe9f198 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:31.214798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623f69e-1000-4a74-81d6-75a82fe9f198 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:31.220031 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 602/1219] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:57:31 2022] GET /identity/v3/role_assignments?user.id=6074872303d3468fb566378fa34e8bea&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:33.350930 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56605290-1c0b-4db5-b2e4-2adcf64ef7d4 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:33.351489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56605290-1c0b-4db5-b2e4-2adcf64ef7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:33.351489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56605290-1c0b-4db5-b2e4-2adcf64ef7d4 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:33.352823 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 618/1220] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:57:33.358144 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46896837-482c-4f5f-8170-f1ff5db116aa None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:33.358542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46896837-482c-4f5f-8170-f1ff5db116aa None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:33.359802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46896837-482c-4f5f-8170-f1ff5db116aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:33.376911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-46896837-482c-4f5f-8170-f1ff5db116aa None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:57:33.378218 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-46896837-482c-4f5f-8170-f1ff5db116aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:57:33.444876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 603/1221] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:57:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:57:33.716485 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56605290-1c0b-4db5-b2e4-2adcf64ef7d4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:33.814884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56605290-1c0b-4db5-b2e4-2adcf64ef7d4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:33.847451 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1e613-4348-4311-b88d-1df1dfecf4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:33.848067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1e613-4348-4311-b88d-1df1dfecf4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:33.848332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1e613-4348-4311-b88d-1df1dfecf4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:33.848591 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1e613-4348-4311-b88d-1df1dfecf4a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:33.858259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1e613-4348-4311-b88d-1df1dfecf4a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:57:33.862974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-41e1e613-4348-4311-b88d-1df1dfecf4a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:57:33.862974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:33.862974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:33.862974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:33.862974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:33.862974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:33.862974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:33.862974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:33.862974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:33.862974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:33.862974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:33.862974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:33.862974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:57:33.862974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:57:33.862974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:57:33.862974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:57:33.865638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:57:33.865638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:57:33.865638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:57:33.865638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:57:33.865638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:33.865638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:33.865638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:33.865638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:33.865638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:33.865638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:33.865638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:33.865638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:33.865638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:33.865638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:33.865638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:33.865638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:33.866677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:33.866677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:33.866677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:33.866677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:33.866677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:57:33.866677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:57:33.866677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:57:33.866677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:57:33.866677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:57:33.866677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:57:33.866677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:57:33.866677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:33.866677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 619/1222] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:57:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:33.870363 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46896837-482c-4f5f-8170-f1ff5db116aa None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:33.967409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46896837-482c-4f5f-8170-f1ff5db116aa None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:33.990047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:34.000727 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6ae28-0296-4733-9b1a-6ecae3396312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:34.001338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6ae28-0296-4733-9b1a-6ecae3396312 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:34.001616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6ae28-0296-4733-9b1a-6ecae3396312 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:34.002440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6ae28-0296-4733-9b1a-6ecae3396312 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:34.003059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6ae28-0296-4733-9b1a-6ecae3396312 None admin] RBAC: Adding query filter params (name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:34.003361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6ae28-0296-4733-9b1a-6ecae3396312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:34.004800 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6ae28-0296-4733-9b1a-6ecae3396312 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:34.012734 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 604/1223] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:57:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:34.020209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1e613-4348-4311-b88d-1df1dfecf4a9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:34.063838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:34.117573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1e613-4348-4311-b88d-1df1dfecf4a9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:34.151471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da70ef-fa20-4759-93db-1dc54d91ff75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:34.152098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da70ef-fa20-4759-93db-1dc54d91ff75 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:34.152419 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da70ef-fa20-4759-93db-1dc54d91ff75 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:34.152691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da70ef-fa20-4759-93db-1dc54d91ff75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:34.162535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da70ef-fa20-4759-93db-1dc54d91ff75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:34.164157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da70ef-fa20-4759-93db-1dc54d91ff75 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:34.178734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-36da70ef-fa20-4759-93db-1dc54d91ff75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:34.178734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:34.178734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:34.178734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:34.178734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:34.178734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:34.178734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:34.178734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:34.178734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:34.178734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:34.178734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:34.178734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:34.178734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:57:34.178734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:57:34.178734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:57:34.178734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:34.179935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:57:34.181076 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:57:34.181076 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:34.181076 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:34.181076 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 620/1224] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:34.185599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6ae28-0296-4733-9b1a-6ecae3396312 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:34.283195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6ae28-0296-4733-9b1a-6ecae3396312 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:34.321028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90e954-99c6-4fbf-a819-ea9317dc1f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:34.321625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90e954-99c6-4fbf-a819-ea9317dc1f50 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:34.321902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90e954-99c6-4fbf-a819-ea9317dc1f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:34.322172 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90e954-99c6-4fbf-a819-ea9317dc1f50 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:34.322802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90e954-99c6-4fbf-a819-ea9317dc1f50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:34.323133 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90e954-99c6-4fbf-a819-ea9317dc1f50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:34.324572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90e954-99c6-4fbf-a819-ea9317dc1f50 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:34.336565 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 605/1225] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:34.342308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da70ef-fa20-4759-93db-1dc54d91ff75 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:34.436815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:34.444095 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da70ef-fa20-4759-93db-1dc54d91ff75 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:34.476465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f883a-1984-445d-b117-d312f8caa8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:34.477079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f883a-1984-445d-b117-d312f8caa8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:34.477355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f883a-1984-445d-b117-d312f8caa8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:34.477621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f883a-1984-445d-b117-d312f8caa8d8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:34.484365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f883a-1984-445d-b117-d312f8caa8d8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:34.485800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f883a-1984-445d-b117-d312f8caa8d8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:34.497892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ae4f883a-1984-445d-b117-d312f8caa8d8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 07:57:34.497892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:34.497892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:34.497892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:34.497892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:34.497892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:34.497892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:34.497892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:34.497892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:34.497892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:34.497892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:34.497892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:34.497892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:57:34.497892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:57:34.497892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:57:34.497892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:34.499048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:34.500229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:34.500229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:34.500229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:57:34.500229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:57:34.500229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:57:34.500229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:57:34.500229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:57:34.500229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:57:34.500229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 07:57:34.500229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:34.500229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 621/1226] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:34.505078 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90e954-99c6-4fbf-a819-ea9317dc1f50 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:34.551220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:34.602252 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90e954-99c6-4fbf-a819-ea9317dc1f50 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:34.636371 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbda7a3-eaca-445b-973f-8ba60b74dbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:34.636955 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbda7a3-eaca-445b-973f-8ba60b74dbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:34.637228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbda7a3-eaca-445b-973f-8ba60b74dbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:34.637492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbda7a3-eaca-445b-973f-8ba60b74dbb3 None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:34.638203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbda7a3-eaca-445b-973f-8ba60b74dbb3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:34.638495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbda7a3-eaca-445b-973f-8ba60b74dbb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:34.639932 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbda7a3-eaca-445b-973f-8ba60b74dbb3 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:34.656190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 606/1227] 10.222.0.30 () {56 vars in 1143 bytes} [Thu Oct 6 07:57:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:34.662783 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f883a-1984-445d-b117-d312f8caa8d8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:34.761927 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f883a-1984-445d-b117-d312f8caa8d8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:34.794221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770eba3-5362-439d-bb50-fe3730515139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:34.794830 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770eba3-5362-439d-bb50-fe3730515139 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:34.795136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770eba3-5362-439d-bb50-fe3730515139 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:34.795405 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770eba3-5362-439d-bb50-fe3730515139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:34.806268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3770eba3-5362-439d-bb50-fe3730515139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:34.807696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3770eba3-5362-439d-bb50-fe3730515139 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:34.821501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3770eba3-5362-439d-bb50-fe3730515139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:34.821501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:34.821501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:34.821501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:34.821501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:34.821501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:34.821501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:34.821501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:34.821501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:34.821501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:34.821501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:34.821501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:34.821501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:57:34.821501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:57:34.821501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:57:34.821501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:34.822648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:57:34.823899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:57:34.823899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:34.823899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:34.823899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 622/1228] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:34.828361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbda7a3-eaca-445b-973f-8ba60b74dbb3 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:34.925685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbda7a3-eaca-445b-973f-8ba60b74dbb3 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:34.927983 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:34.958067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5064977-2118-45c3-b647-cbb1b03221cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:34.958659 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5064977-2118-45c3-b647-cbb1b03221cb None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:34.958928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5064977-2118-45c3-b647-cbb1b03221cb None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:34.959223 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5064977-2118-45c3-b647-cbb1b03221cb None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:34.959850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5064977-2118-45c3-b647-cbb1b03221cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:34.960145 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5064977-2118-45c3-b647-cbb1b03221cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:34.961501 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5064977-2118-45c3-b647-cbb1b03221cb None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:34.972736 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 607/1229] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:34.978585 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770eba3-5362-439d-bb50-fe3730515139 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:34.987977 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:35.077801 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770eba3-5362-439d-bb50-fe3730515139 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:35.111034 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05492b79-0293-4fcd-ab3d-071664f8f273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:35.111856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05492b79-0293-4fcd-ab3d-071664f8f273 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:35.112125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05492b79-0293-4fcd-ab3d-071664f8f273 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:35.112397 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05492b79-0293-4fcd-ab3d-071664f8f273 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:35.118874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05492b79-0293-4fcd-ab3d-071664f8f273 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:35.120367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05492b79-0293-4fcd-ab3d-071664f8f273 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:35.132239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-05492b79-0293-4fcd-ab3d-071664f8f273 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:57:35.132239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:35.132239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:35.132239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:35.132239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:35.132239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:35.132239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:35.132239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:35.132239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:35.132239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:35.132239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:35.132239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:35.132239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:57:35.132239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:57:35.132239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:35.133337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:35.134418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:57:35.134418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:57:35.134418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:57:35.134418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:57:35.134418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:57:35.134418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:57:35.134418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:57:35.134418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:35.134418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 623/1230] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:57:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:35.139288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5064977-2118-45c3-b647-cbb1b03221cb None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:35.242771 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5064977-2118-45c3-b647-cbb1b03221cb None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:35.277263 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fc6ce-e890-4f7e-a0d2-d082c6b73ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:35.277907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fc6ce-e890-4f7e-a0d2-d082c6b73ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:35.278187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fc6ce-e890-4f7e-a0d2-d082c6b73ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:35.278460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fc6ce-e890-4f7e-a0d2-d082c6b73ff4 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:35.279248 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fc6ce-e890-4f7e-a0d2-d082c6b73ff4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:35.279552 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fc6ce-e890-4f7e-a0d2-d082c6b73ff4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:35.280991 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fc6ce-e890-4f7e-a0d2-d082c6b73ff4 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:35.292720 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 608/1231] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:57:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:35.298926 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05492b79-0293-4fcd-ab3d-071664f8f273 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:35.371005 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:35.405512 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05492b79-0293-4fcd-ab3d-071664f8f273 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:35.438905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88d4a2-029e-4b95-98a0-18628c636f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:35.440448 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88d4a2-029e-4b95-98a0-18628c636f25 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:35.440719 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88d4a2-029e-4b95-98a0-18628c636f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:35.440989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88d4a2-029e-4b95-98a0-18628c636f25 None admin] PATH_INFO: `/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/6074872303d3468fb566378fa34e8bea/roles/9f0e36c2ed8843fbabd3a4cf30bc3725` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:35.455844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88d4a2-029e-4b95-98a0-18628c636f25 None admin] RBAC: Authorizing `identity:create_grant(project_id=399f0095cc8b4c64893ccc75af13c52d, user_id=6074872303d3468fb566378fa34e8bea, role_id=9f0e36c2ed8843fbabd3a4cf30bc3725)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:35.457282 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88d4a2-029e-4b95-98a0-18628c636f25 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:35.538062 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 624/1232] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:57:35 2022] PUT /identity/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/6074872303d3468fb566378fa34e8bea/roles/9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:57:37.804177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143505f5-7a03-4507-b613-fdeb37266c36 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:37.804744 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143505f5-7a03-4507-b613-fdeb37266c36 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:37.804744 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143505f5-7a03-4507-b613-fdeb37266c36 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:37.806075 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 609/1233] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:57:37.811378 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ae52d-95a4-4352-8fd4-8d9ee15c94d2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:37.814892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ae52d-95a4-4352-8fd4-8d9ee15c94d2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:37.814892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ae52d-95a4-4352-8fd4-8d9ee15c94d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:37.829417 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6a2ae52d-95a4-4352-8fd4-8d9ee15c94d2 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:57:37.831137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6a2ae52d-95a4-4352-8fd4-8d9ee15c94d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:57:37.894869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 625/1234] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:57:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:57:38.162654 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143505f5-7a03-4507-b613-fdeb37266c36 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:38.187590 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:38.263358 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143505f5-7a03-4507-b613-fdeb37266c36 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:38.296440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05621af-077d-4581-8893-2b01ce925845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:38.303900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05621af-077d-4581-8893-2b01ce925845 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:38.303900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05621af-077d-4581-8893-2b01ce925845 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:38.303900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05621af-077d-4581-8893-2b01ce925845 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:38.307652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05621af-077d-4581-8893-2b01ce925845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:57:38.312483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-e05621af-077d-4581-8893-2b01ce925845 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:57:38.312483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:38.312483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:38.312483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:38.312483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:38.312483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:38.312483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:38.312483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:38.312483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:38.312483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:38.312483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:38.312483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:38.312483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:57:38.312483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:57:38.312483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:57:38.312483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:57:38.313680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:57:38.313680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:57:38.313680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:57:38.313680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:57:38.313680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:38.313680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:38.313680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:38.313680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:38.313680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:38.313680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:38.313680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:38.313680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:38.313680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:38.313680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:38.313680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:38.313680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:38.315685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:38.315685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:38.315685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:38.315685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:38.315685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:57:38.315685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:57:38.315685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:57:38.315685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:57:38.315685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:57:38.315685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:57:38.315685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 06 07:57:38.315685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:57:38.315685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 610/1235] 10.222.0.30 () {56 vars in 1102 bytes} [Thu Oct 6 07:57:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:38.320281 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ae52d-95a4-4352-8fd4-8d9ee15c94d2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:38.365163 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:38.416743 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ae52d-95a4-4352-8fd4-8d9ee15c94d2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:38.450224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307fad3-6c91-41dc-b5f2-b7f318e768b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:38.450821 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307fad3-6c91-41dc-b5f2-b7f318e768b3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:38.451136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307fad3-6c91-41dc-b5f2-b7f318e768b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:38.451409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307fad3-6c91-41dc-b5f2-b7f318e768b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:38.452068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307fad3-6c91-41dc-b5f2-b7f318e768b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:38.452368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307fad3-6c91-41dc-b5f2-b7f318e768b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:38.453752 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307fad3-6c91-41dc-b5f2-b7f318e768b3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:38.462383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 626/1236] 10.222.0.30 () {56 vars in 1103 bytes} [Thu Oct 6 07:57:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:38.468004 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05621af-077d-4581-8893-2b01ce925845 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:38.553673 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:38.569814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05621af-077d-4581-8893-2b01ce925845 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:38.603156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f7143-933f-4c46-8cc6-10c1860c0b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:38.603796 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f7143-933f-4c46-8cc6-10c1860c0b13 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:38.604074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f7143-933f-4c46-8cc6-10c1860c0b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:38.604345 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f7143-933f-4c46-8cc6-10c1860c0b13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:38.616012 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f7143-933f-4c46-8cc6-10c1860c0b13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:38.617489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f7143-933f-4c46-8cc6-10c1860c0b13 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:38.633965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-484f7143-933f-4c46-8cc6-10c1860c0b13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:38.633965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:38.633965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:38.633965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:38.633965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:38.633965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:38.633965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:38.633965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:38.633965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:38.633965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:38.633965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:38.633965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:38.633965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:57:38.633965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:57:38.633965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:57:38.633965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:38.635049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:57:38.636359 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:57:38.636359 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:38.636359 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:57:38.636359 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 611/1237] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:38.640979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307fad3-6c91-41dc-b5f2-b7f318e768b3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:38.738404 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307fad3-6c91-41dc-b5f2-b7f318e768b3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:38.746897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:38.771284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a1db7-8f35-4986-8185-fa8603a44c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:38.771876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a1db7-8f35-4986-8185-fa8603a44c11 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:38.772153 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a1db7-8f35-4986-8185-fa8603a44c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:38.772424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a1db7-8f35-4986-8185-fa8603a44c11 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:38.773058 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a1db7-8f35-4986-8185-fa8603a44c11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:38.773358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a1db7-8f35-4986-8185-fa8603a44c11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:38.774736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a1db7-8f35-4986-8185-fa8603a44c11 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:38.786006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 627/1238] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:57:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:38.791858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f7143-933f-4c46-8cc6-10c1860c0b13 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:38.891479 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f7143-933f-4c46-8cc6-10c1860c0b13 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:38.924383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66e609-d1ac-4337-aab0-61ccbd03c07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:38.924991 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66e609-d1ac-4337-aab0-61ccbd03c07d None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:38.925263 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66e609-d1ac-4337-aab0-61ccbd03c07d None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:38.925529 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66e609-d1ac-4337-aab0-61ccbd03c07d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:38.928114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:38.932859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66e609-d1ac-4337-aab0-61ccbd03c07d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:38.934290 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66e609-d1ac-4337-aab0-61ccbd03c07d None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:38.945695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-2f66e609-d1ac-4337-aab0-61ccbd03c07d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 07:57:38.945695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:38.945695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:38.945695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:38.945695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:38.945695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:38.945695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:38.945695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:38.945695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:38.945695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:38.945695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:38.945695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:38.945695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:57:38.945695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:57:38.945695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:57:38.945695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:38.946711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:38.947735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:38.947735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:38.947735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:57:38.947735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:57:38.947735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:57:38.947735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:57:38.947735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:57:38.947735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:57:38.947735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 07:57:38.947735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:57:38.947735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 612/1239] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:38.952767 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a1db7-8f35-4986-8185-fa8603a44c11 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:39.052054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a1db7-8f35-4986-8185-fa8603a44c11 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:39.083904 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8be081-8573-439c-a663-9f307a4b8ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:39.085230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8be081-8573-439c-a663-9f307a4b8ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:39.085230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8be081-8573-439c-a663-9f307a4b8ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:39.085230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8be081-8573-439c-a663-9f307a4b8ad9 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:39.085770 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8be081-8573-439c-a663-9f307a4b8ad9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:39.086067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8be081-8573-439c-a663-9f307a4b8ad9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:39.087574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8be081-8573-439c-a663-9f307a4b8ad9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:39.107180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 628/1240] 10.222.0.30 () {56 vars in 1143 bytes} [Thu Oct 6 07:57:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:39.113636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66e609-d1ac-4337-aab0-61ccbd03c07d None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:39.212069 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66e609-d1ac-4337-aab0-61ccbd03c07d None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:39.245571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31023258-32b7-4792-8241-5c7055c17983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:39.246169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31023258-32b7-4792-8241-5c7055c17983 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:39.246445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31023258-32b7-4792-8241-5c7055c17983 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:39.246715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31023258-32b7-4792-8241-5c7055c17983 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:39.306851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:39.312974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31023258-32b7-4792-8241-5c7055c17983 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:39.314618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31023258-32b7-4792-8241-5c7055c17983 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:39.409856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 613/1241] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:57:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 06 07:57:39.415861 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8be081-8573-439c-a663-9f307a4b8ad9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:39.444838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:39.515891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8be081-8573-439c-a663-9f307a4b8ad9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:39.549876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72515b-39b5-4856-a339-185bb963ee97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:39.550488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72515b-39b5-4856-a339-185bb963ee97 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:39.550760 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72515b-39b5-4856-a339-185bb963ee97 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:39.551119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72515b-39b5-4856-a339-185bb963ee97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:39.560917 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72515b-39b5-4856-a339-185bb963ee97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:39.562314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72515b-39b5-4856-a339-185bb963ee97 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:39.576836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5f72515b-39b5-4856-a339-185bb963ee97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:39.576836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:39.576836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:39.576836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:39.576836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:39.576836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:39.576836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:39.576836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:39.576836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:39.576836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:39.576836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:39.576836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:39.576836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:57:39.576836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:57:39.576836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:57:39.576836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:39.577964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:57:39.579085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:57:39.579085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:39.579085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:39.579085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 629/1242] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:39.584266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31023258-32b7-4792-8241-5c7055c17983 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:39.677711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:39.683282 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31023258-32b7-4792-8241-5c7055c17983 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:39.717924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3a90c-42b7-40d0-9dcf-e9a9249cd9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:39.718520 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3a90c-42b7-40d0-9dcf-e9a9249cd9cb None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:39.718795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3a90c-42b7-40d0-9dcf-e9a9249cd9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:39.719097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3a90c-42b7-40d0-9dcf-e9a9249cd9cb None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:39.719727 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3a90c-42b7-40d0-9dcf-e9a9249cd9cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:39.720023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3a90c-42b7-40d0-9dcf-e9a9249cd9cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:39.721962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3a90c-42b7-40d0-9dcf-e9a9249cd9cb None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:39.732703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 614/1243] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:39.739037 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72515b-39b5-4856-a339-185bb963ee97 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:39.816659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:39.836969 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72515b-39b5-4856-a339-185bb963ee97 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:39.870818 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190acb3-0e2c-4c61-a58e-de9bc5d1d799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:39.871458 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190acb3-0e2c-4c61-a58e-de9bc5d1d799 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:39.871731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190acb3-0e2c-4c61-a58e-de9bc5d1d799 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:39.872266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190acb3-0e2c-4c61-a58e-de9bc5d1d799 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:39.878690 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190acb3-0e2c-4c61-a58e-de9bc5d1d799 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:39.880167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190acb3-0e2c-4c61-a58e-de9bc5d1d799 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:39.890903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1190acb3-0e2c-4c61-a58e-de9bc5d1d799 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:57:39.890903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:39.890903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:39.890903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:39.890903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:39.890903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:39.890903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:39.890903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:39.890903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:39.890903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:39.890903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:39.890903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:39.890903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:57:39.890903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:57:39.890903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:39.892162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:39.899552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:57:39.899552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:57:39.899552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:57:39.899552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:57:39.899552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:57:39.899552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:57:39.899552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:57:39.899552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:39.899552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 630/1244] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:57:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:39.900189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3a90c-42b7-40d0-9dcf-e9a9249cd9cb None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:39.995158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3a90c-42b7-40d0-9dcf-e9a9249cd9cb None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:40.027080 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc31f9-cf49-48c1-a127-a90c22a12595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:40.027668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc31f9-cf49-48c1-a127-a90c22a12595 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:40.027938 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc31f9-cf49-48c1-a127-a90c22a12595 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:40.028203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc31f9-cf49-48c1-a127-a90c22a12595 None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:40.028881 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc31f9-cf49-48c1-a127-a90c22a12595 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:40.029175 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc31f9-cf49-48c1-a127-a90c22a12595 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:40.030588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc31f9-cf49-48c1-a127-a90c22a12595 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:40.042745 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 615/1245] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:57:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:40.049573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190acb3-0e2c-4c61-a58e-de9bc5d1d799 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:40.146707 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190acb3-0e2c-4c61-a58e-de9bc5d1d799 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:40.180381 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e7596-1126-4588-a36f-a6957704116b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:40.180998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e7596-1126-4588-a36f-a6957704116b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:40.181272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e7596-1126-4588-a36f-a6957704116b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:40.181538 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e7596-1126-4588-a36f-a6957704116b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:40.182249 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e7596-1126-4588-a36f-a6957704116b None admin] RBAC: Adding query filter params (role.id=9f0e36c2ed8843fbabd3a4cf30bc3725, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=6074872303d3468fb566378fa34e8bea) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:40.182551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e7596-1126-4588-a36f-a6957704116b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:40.183994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e7596-1126-4588-a36f-a6957704116b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:40.189683 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 631/1246] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:57:40 2022] GET /identity/v3/role_assignments?user.id=6074872303d3468fb566378fa34e8bea&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=9f0e36c2ed8843fbabd3a4cf30bc3725 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:42.378072 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fab298-59e2-4680-89b4-583e4fec9deb None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:42.378591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fab298-59e2-4680-89b4-583e4fec9deb None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:42.378591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fab298-59e2-4680-89b4-583e4fec9deb None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:42.379987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 616/1247] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:57:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 07:57:42.385256 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4939f4-a4b3-4b92-b9bd-64ded949d92c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:42.385656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4939f4-a4b3-4b92-b9bd-64ded949d92c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:42.385855 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4939f4-a4b3-4b92-b9bd-64ded949d92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:42.392752 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:42.404931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ca4939f4-a4b3-4b92-b9bd-64ded949d92c None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:57:42.406035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ca4939f4-a4b3-4b92-b9bd-64ded949d92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:57:42.471901 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 632/1248] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:57:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:57:42.740082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fab298-59e2-4680-89b4-583e4fec9deb None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:42.758176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:42.837797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fab298-59e2-4680-89b4-583e4fec9deb None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:42.870902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669e583-c8f1-485b-b1ba-2b81192ec9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:42.871536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669e583-c8f1-485b-b1ba-2b81192ec9fe None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:42.871811 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669e583-c8f1-485b-b1ba-2b81192ec9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:42.872333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669e583-c8f1-485b-b1ba-2b81192ec9fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:42.883360 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669e583-c8f1-485b-b1ba-2b81192ec9fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:57:42.888079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-7669e583-c8f1-485b-b1ba-2b81192ec9fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:57:42.888079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:42.888079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:42.888079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:42.888079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:42.888079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:42.888079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:42.888079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:42.888079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:42.888079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:42.888079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:42.888079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:42.888079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:57:42.888079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:57:42.888079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:57:42.888079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:57:42.889261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:57:42.889261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:57:42.889261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:57:42.889261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:57:42.889261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:42.889261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:42.889261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:42.889261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:42.889261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:42.889261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:42.889261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:42.889261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:42.889261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:42.889261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:42.889261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:42.889261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:42.890305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:42.890305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:42.890305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:42.890305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:42.890305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:57:42.890305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:57:42.890305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:57:42.890305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:57:42.890305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:57:42.890305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:57:42.890305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:57:42.890305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:57:42.890305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 617/1249] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:57:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:42.895628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4939f4-a4b3-4b92-b9bd-64ded949d92c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:42.992344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4939f4-a4b3-4b92-b9bd-64ded949d92c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:43.021052 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:43.025657 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb5f7b-24e2-4111-ad05-251cd97e0623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:43.026342 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb5f7b-24e2-4111-ad05-251cd97e0623 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:43.026617 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb5f7b-24e2-4111-ad05-251cd97e0623 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:43.026881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb5f7b-24e2-4111-ad05-251cd97e0623 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:43.027558 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb5f7b-24e2-4111-ad05-251cd97e0623 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:43.027850 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb5f7b-24e2-4111-ad05-251cd97e0623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:43.029191 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb5f7b-24e2-4111-ad05-251cd97e0623 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:43.037467 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 633/1250] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:57:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:43.043462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669e583-c8f1-485b-b1ba-2b81192ec9fe None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:43.119313 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:43.142271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669e583-c8f1-485b-b1ba-2b81192ec9fe None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:43.174963 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b16f4-39d8-4a75-ad83-ff9903bb0334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:43.175579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b16f4-39d8-4a75-ad83-ff9903bb0334 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:43.175854 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b16f4-39d8-4a75-ad83-ff9903bb0334 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:43.176123 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b16f4-39d8-4a75-ad83-ff9903bb0334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:43.186171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b16f4-39d8-4a75-ad83-ff9903bb0334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:43.187652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b16f4-39d8-4a75-ad83-ff9903bb0334 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:43.201693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-423b16f4-39d8-4a75-ad83-ff9903bb0334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:43.201693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:43.201693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:43.201693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:43.201693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:43.201693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:43.201693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:43.201693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:43.201693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:43.201693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:43.201693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:43.201693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:43.201693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:57:43.201693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:57:43.201693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:57:43.201693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:43.202842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:57:43.204171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:57:43.204171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:43.204171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:57:43.204171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 618/1251] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:43.208775 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb5f7b-24e2-4111-ad05-251cd97e0623 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:43.307057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb5f7b-24e2-4111-ad05-251cd97e0623 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:43.339543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657a650-5c5e-40b5-a4d1-7549a801ae56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:43.346045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657a650-5c5e-40b5-a4d1-7549a801ae56 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:43.346045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657a650-5c5e-40b5-a4d1-7549a801ae56 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:43.346045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657a650-5c5e-40b5-a4d1-7549a801ae56 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:43.346045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657a650-5c5e-40b5-a4d1-7549a801ae56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:43.346045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657a650-5c5e-40b5-a4d1-7549a801ae56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:43.346045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657a650-5c5e-40b5-a4d1-7549a801ae56 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:43.354207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 634/1252] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:43.360019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b16f4-39d8-4a75-ad83-ff9903bb0334 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:43.470451 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b16f4-39d8-4a75-ad83-ff9903bb0334 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:43.497733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:43.505578 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e096ca-26fb-468b-bb06-1d992f1469fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:43.506201 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e096ca-26fb-468b-bb06-1d992f1469fe None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:43.506481 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e096ca-26fb-468b-bb06-1d992f1469fe None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:43.506755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e096ca-26fb-468b-bb06-1d992f1469fe None admin] PATH_INFO: `/v3/users/placement` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:43.514027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e096ca-26fb-468b-bb06-1d992f1469fe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:43.515510 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e096ca-26fb-468b-bb06-1d992f1469fe None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:43.526490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-01e096ca-26fb-468b-bb06-1d992f1469fe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 07:57:43.526490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:43.526490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:43.526490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:43.526490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:43.526490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:43.526490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:43.526490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:43.526490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:43.526490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:43.526490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:43.526490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:43.526490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:57:43.526490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:57:43.526490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:57:43.526490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:43.527651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:43.529027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:43.529027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:43.529027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:57:43.529027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:57:43.529027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:57:43.529027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:57:43.529027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:57:43.529027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:57:43.529027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 07:57:43.529027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:57:43.529027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 619/1253] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:43.534059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657a650-5c5e-40b5-a4d1-7549a801ae56 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:43.588115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:43.632305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657a650-5c5e-40b5-a4d1-7549a801ae56 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:43.664161 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51799947-dcf9-437b-9a32-5e97d0a7346f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:43.667111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51799947-dcf9-437b-9a32-5e97d0a7346f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:43.667111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51799947-dcf9-437b-9a32-5e97d0a7346f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:43.667111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51799947-dcf9-437b-9a32-5e97d0a7346f None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:43.667111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51799947-dcf9-437b-9a32-5e97d0a7346f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:43.667111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51799947-dcf9-437b-9a32-5e97d0a7346f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:43.667743 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51799947-dcf9-437b-9a32-5e97d0a7346f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:43.684703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 635/1254] 10.222.0.30 () {56 vars in 1143 bytes} [Thu Oct 6 07:57:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:57:43.690689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e096ca-26fb-468b-bb06-1d992f1469fe None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:43.801352 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e096ca-26fb-468b-bb06-1d992f1469fe None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:43.835134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbf106-7023-42db-b701-a039d4a68ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:43.835735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbf106-7023-42db-b701-a039d4a68ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:43.836014 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbf106-7023-42db-b701-a039d4a68ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:43.836330 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbf106-7023-42db-b701-a039d4a68ab8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:43.890266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:43.902988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbf106-7023-42db-b701-a039d4a68ab8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:43.904472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbf106-7023-42db-b701-a039d4a68ab8 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:44.002323 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 620/1255] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:57:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 07:57:44.008409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51799947-dcf9-437b-9a32-5e97d0a7346f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:44.110605 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51799947-dcf9-437b-9a32-5e97d0a7346f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:44.123795 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:44.144291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbba1ef-2510-4094-9bfb-4e1e8499dcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:44.144855 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbba1ef-2510-4094-9bfb-4e1e8499dcec None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:44.145133 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbba1ef-2510-4094-9bfb-4e1e8499dcec None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:44.145401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbba1ef-2510-4094-9bfb-4e1e8499dcec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:44.155590 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbba1ef-2510-4094-9bfb-4e1e8499dcec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:44.157025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbba1ef-2510-4094-9bfb-4e1e8499dcec None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:44.172004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1cbba1ef-2510-4094-9bfb-4e1e8499dcec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:44.172004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:44.172004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:44.172004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:44.172004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:44.172004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:44.172004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:44.172004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:44.172004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:44.172004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:44.172004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:44.172004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:44.172004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:57:44.172004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:57:44.172004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:57:44.172004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:44.173277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:57:44.174268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:57:44.174268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:44.174268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:44.174268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 636/1256] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:44.179169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbf106-7023-42db-b701-a039d4a68ab8 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:44.264201 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:44.275943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbf106-7023-42db-b701-a039d4a68ab8 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:44.308054 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b213ea4-ee24-4320-9516-21c7f0349fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:44.308652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b213ea4-ee24-4320-9516-21c7f0349fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:44.308929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b213ea4-ee24-4320-9516-21c7f0349fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:44.309233 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b213ea4-ee24-4320-9516-21c7f0349fd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:44.309865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b213ea4-ee24-4320-9516-21c7f0349fd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:44.310168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b213ea4-ee24-4320-9516-21c7f0349fd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:44.311560 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b213ea4-ee24-4320-9516-21c7f0349fd6 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:44.323578 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 621/1257] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:44.330006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbba1ef-2510-4094-9bfb-4e1e8499dcec None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:44.434923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbba1ef-2510-4094-9bfb-4e1e8499dcec None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:44.468189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ec1cf-c451-4fac-ae23-19bb4592428d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:44.468804 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ec1cf-c451-4fac-ae23-19bb4592428d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:44.469077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ec1cf-c451-4fac-ae23-19bb4592428d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:44.469345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ec1cf-c451-4fac-ae23-19bb4592428d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:44.475678 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ec1cf-c451-4fac-ae23-19bb4592428d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:44.477108 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ec1cf-c451-4fac-ae23-19bb4592428d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:44.488059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-869ec1cf-c451-4fac-ae23-19bb4592428d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:57:44.488059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:44.488059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:44.488059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:44.488059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:44.488059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:44.488059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:44.488059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:44.488059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:44.488059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:44.488059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:44.488059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:44.488059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:57:44.488059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:57:44.488059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:44.489102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:44.490038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:57:44.490038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:57:44.490038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:57:44.490038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:57:44.490038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:57:44.490038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:57:44.490038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:57:44.490038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:44.490038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 637/1258] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:57:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:44.495186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b213ea4-ee24-4320-9516-21c7f0349fd6 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:44.592806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b213ea4-ee24-4320-9516-21c7f0349fd6 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:44.628726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110e9f9-3eec-410b-8233-ad5df6c7139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:44.629317 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110e9f9-3eec-410b-8233-ad5df6c7139c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:44.629589 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110e9f9-3eec-410b-8233-ad5df6c7139c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:44.629855 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110e9f9-3eec-410b-8233-ad5df6c7139c None admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:44.630561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110e9f9-3eec-410b-8233-ad5df6c7139c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:44.630857 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110e9f9-3eec-410b-8233-ad5df6c7139c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:44.632308 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110e9f9-3eec-410b-8233-ad5df6c7139c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:44.644006 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 622/1259] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:57:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:44.650033 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ec1cf-c451-4fac-ae23-19bb4592428d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:44.677205 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:44.749323 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ec1cf-c451-4fac-ae23-19bb4592428d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:44.784059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd871f-5a97-4446-8c70-bdbea2db58dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:44.784656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd871f-5a97-4446-8c70-bdbea2db58dd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:44.784935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd871f-5a97-4446-8c70-bdbea2db58dd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:44.785209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd871f-5a97-4446-8c70-bdbea2db58dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:44.786000 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd871f-5a97-4446-8c70-bdbea2db58dd None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=6074872303d3468fb566378fa34e8bea) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:44.786303 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd871f-5a97-4446-8c70-bdbea2db58dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:44.787750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd871f-5a97-4446-8c70-bdbea2db58dd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:44.793012 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 638/1260] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:57:44 2022] GET /identity/v3/role_assignments?user.id=6074872303d3468fb566378fa34e8bea&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:46.944463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73c451-af7b-4d00-9795-2f27e5ed6dc0 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:46.945060 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73c451-af7b-4d00-9795-2f27e5ed6dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:46.945060 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73c451-af7b-4d00-9795-2f27e5ed6dc0 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:46.946356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 623/1261] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:57:46.951408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eee88-d672-4308-a115-a95743690067 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:46.951815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eee88-d672-4308-a115-a95743690067 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:46.951918 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eee88-d672-4308-a115-a95743690067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:46.970730 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-478eee88-d672-4308-a115-a95743690067 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:57:46.971929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-478eee88-d672-4308-a115-a95743690067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:57:47.037368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 639/1262] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:57:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:57:47.307855 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73c451-af7b-4d00-9795-2f27e5ed6dc0 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:47.319090 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:47.406657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73c451-af7b-4d00-9795-2f27e5ed6dc0 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:47.441045 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473d04e-9752-4c0e-ae14-7fa111e105d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:47.441659 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473d04e-9752-4c0e-ae14-7fa111e105d5 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:47.441938 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473d04e-9752-4c0e-ae14-7fa111e105d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:47.442208 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473d04e-9752-4c0e-ae14-7fa111e105d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:47.452057 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473d04e-9752-4c0e-ae14-7fa111e105d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:57:47.457109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-a473d04e-9752-4c0e-ae14-7fa111e105d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:57:47.457109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:47.457109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:47.457109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:47.457109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:47.457109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:47.457109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:47.457109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:47.457109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:47.457109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:47.457109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:47.457109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:47.457109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:57:47.457109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:57:47.457109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:57:47.457109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:57:47.458243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:57:47.458243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:57:47.458243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:57:47.458243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:57:47.458243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:47.458243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:47.458243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:47.458243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:47.458243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:47.458243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:47.458243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:47.458243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:47.458243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:47.458243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:47.458243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:47.458243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:47.459337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:47.459337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:47.459337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:47.459337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:47.459337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:57:47.459337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:57:47.459337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:57:47.459337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:57:47.459337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:57:47.459337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:57:47.459337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:57:47.459337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:57:47.459337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 624/1263] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:57:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:47.465542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478eee88-d672-4308-a115-a95743690067 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:47.489185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:47.567453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478eee88-d672-4308-a115-a95743690067 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:47.601158 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31791c21-d7b3-4408-8f39-217d82ead913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:47.601746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31791c21-d7b3-4408-8f39-217d82ead913 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:47.602019 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31791c21-d7b3-4408-8f39-217d82ead913 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:47.602282 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31791c21-d7b3-4408-8f39-217d82ead913 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:47.602933 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31791c21-d7b3-4408-8f39-217d82ead913 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:47.603259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31791c21-d7b3-4408-8f39-217d82ead913 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:47.604654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31791c21-d7b3-4408-8f39-217d82ead913 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:47.612784 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 640/1264] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:57:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:47.618629 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473d04e-9752-4c0e-ae14-7fa111e105d5 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:47.684806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:47.717864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473d04e-9752-4c0e-ae14-7fa111e105d5 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:47.752857 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba88c5-4c7b-4c49-a6e0-d3f629f5b733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:47.753481 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba88c5-4c7b-4c49-a6e0-d3f629f5b733 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:47.753756 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba88c5-4c7b-4c49-a6e0-d3f629f5b733 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:47.754024 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba88c5-4c7b-4c49-a6e0-d3f629f5b733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:47.763574 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba88c5-4c7b-4c49-a6e0-d3f629f5b733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:47.765131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba88c5-4c7b-4c49-a6e0-d3f629f5b733 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:47.780448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-1dba88c5-4c7b-4c49-a6e0-d3f629f5b733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:47.780448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:47.780448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:47.780448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:47.780448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:47.780448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:47.780448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:47.780448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:47.780448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:47.780448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:47.780448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:47.780448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:47.780448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:57:47.780448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:57:47.780448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:57:47.780448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:47.783747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:57:47.784901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:57:47.784901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:47.784901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:57:47.784901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 625/1265] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:47.788232 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31791c21-d7b3-4408-8f39-217d82ead913 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:47.867788 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:47.886967 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31791c21-d7b3-4408-8f39-217d82ead913 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:47.919189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582b2b0-af81-4911-8f9a-1117efd4e30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:47.919783 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582b2b0-af81-4911-8f9a-1117efd4e30b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:47.920055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582b2b0-af81-4911-8f9a-1117efd4e30b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:47.920346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582b2b0-af81-4911-8f9a-1117efd4e30b None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:47.920973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582b2b0-af81-4911-8f9a-1117efd4e30b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:47.921291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582b2b0-af81-4911-8f9a-1117efd4e30b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:47.923239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582b2b0-af81-4911-8f9a-1117efd4e30b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:47.934129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 641/1266] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:47.940617 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba88c5-4c7b-4c49-a6e0-d3f629f5b733 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:48.039452 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba88c5-4c7b-4c49-a6e0-d3f629f5b733 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:48.061176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:48.076063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02083eb-2d56-49fa-8518-9da7eb2f6c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:48.076735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02083eb-2d56-49fa-8518-9da7eb2f6c55 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:48.077018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02083eb-2d56-49fa-8518-9da7eb2f6c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:48.077285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02083eb-2d56-49fa-8518-9da7eb2f6c55 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:48.085471 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02083eb-2d56-49fa-8518-9da7eb2f6c55 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:48.086898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02083eb-2d56-49fa-8518-9da7eb2f6c55 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:48.098724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-d02083eb-2d56-49fa-8518-9da7eb2f6c55 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 07:57:48.098724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:48.098724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:48.098724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:48.098724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:48.098724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:48.098724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:48.098724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:48.098724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:48.098724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:48.098724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:48.098724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:48.098724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:57:48.098724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:57:48.098724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:57:48.098724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:48.099943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:48.101222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:48.101222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:48.101222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:57:48.101222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:57:48.101222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:57:48.101222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:57:48.101222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:57:48.101222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:57:48.101222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 07:57:48.101222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:57:48.101222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 626/1267] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:48.105687 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582b2b0-af81-4911-8f9a-1117efd4e30b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:48.206964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582b2b0-af81-4911-8f9a-1117efd4e30b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:48.249954 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22944c8-08e6-4cc7-843d-12ebb651f1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:48.250544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22944c8-08e6-4cc7-843d-12ebb651f1c2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:48.250844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22944c8-08e6-4cc7-843d-12ebb651f1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:48.251633 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22944c8-08e6-4cc7-843d-12ebb651f1c2 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:48.252350 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22944c8-08e6-4cc7-843d-12ebb651f1c2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:48.252651 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22944c8-08e6-4cc7-843d-12ebb651f1c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:48.254075 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22944c8-08e6-4cc7-843d-12ebb651f1c2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:48.262867 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:48.270858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 642/1268] 10.222.0.30 () {56 vars in 1143 bytes} [Thu Oct 6 07:57:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:48.276678 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02083eb-2d56-49fa-8518-9da7eb2f6c55 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:48.376567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02083eb-2d56-49fa-8518-9da7eb2f6c55 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:48.410259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba977e6-8a0d-4b65-a143-9969f32f1f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:48.410806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba977e6-8a0d-4b65-a143-9969f32f1f3c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:48.411111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba977e6-8a0d-4b65-a143-9969f32f1f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:48.411381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba977e6-8a0d-4b65-a143-9969f32f1f3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:48.421561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba977e6-8a0d-4b65-a143-9969f32f1f3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:48.423145 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba977e6-8a0d-4b65-a143-9969f32f1f3c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:48.437099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-3ba977e6-8a0d-4b65-a143-9969f32f1f3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:48.437099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:48.437099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:48.437099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:48.437099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:48.437099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:48.437099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:48.437099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:48.437099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:48.437099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:48.437099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:48.437099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:48.437099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:57:48.437099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:57:48.437099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:57:48.437099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:48.441198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:57:48.442231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:57:48.442231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:48.442231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:57:48.442231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 627/1269] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:48.444082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22944c8-08e6-4cc7-843d-12ebb651f1c2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:48.540319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22944c8-08e6-4cc7-843d-12ebb651f1c2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:48.572020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec816fb-e205-4e71-8201-4c566b04d6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:48.572600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec816fb-e205-4e71-8201-4c566b04d6ab None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:48.572871 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec816fb-e205-4e71-8201-4c566b04d6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:48.573137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec816fb-e205-4e71-8201-4c566b04d6ab None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:48.573763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec816fb-e205-4e71-8201-4c566b04d6ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:48.574089 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec816fb-e205-4e71-8201-4c566b04d6ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:48.575437 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec816fb-e205-4e71-8201-4c566b04d6ab None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:48.586633 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 643/1270] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:57:48.592236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba977e6-8a0d-4b65-a143-9969f32f1f3c None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:48.623527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:48.693029 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba977e6-8a0d-4b65-a143-9969f32f1f3c None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:48.726317 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3eea39-d58c-4ee4-bc5f-5143c3d9e055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:48.733975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3eea39-d58c-4ee4-bc5f-5143c3d9e055 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:48.733975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3eea39-d58c-4ee4-bc5f-5143c3d9e055 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:48.733975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3eea39-d58c-4ee4-bc5f-5143c3d9e055 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:48.733975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3eea39-d58c-4ee4-bc5f-5143c3d9e055 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:48.735678 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3eea39-d58c-4ee4-bc5f-5143c3d9e055 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:48.746432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-fb3eea39-d58c-4ee4-bc5f-5143c3d9e055 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:57:48.746432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:48.746432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:48.746432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:48.746432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:48.746432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:48.746432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:48.746432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:48.746432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:48.746432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:48.746432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:48.746432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:48.746432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:57:48.746432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:57:48.746432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:48.747862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:48.749101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:57:48.749101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:57:48.749101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:57:48.749101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:57:48.749101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:57:48.749101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:57:48.749101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:57:48.749101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:57:48.749101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 628/1271] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:57:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:48.754648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec816fb-e205-4e71-8201-4c566b04d6ab None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:48.811733 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:48.853656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec816fb-e205-4e71-8201-4c566b04d6ab None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:48.886239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c66a2-b28c-43df-9908-1d66823d35de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:48.888450 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c66a2-b28c-43df-9908-1d66823d35de None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:48.888450 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c66a2-b28c-43df-9908-1d66823d35de None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:48.888450 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c66a2-b28c-43df-9908-1d66823d35de None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:48.888450 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c66a2-b28c-43df-9908-1d66823d35de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:48.888450 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c66a2-b28c-43df-9908-1d66823d35de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:48.889723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c66a2-b28c-43df-9908-1d66823d35de None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:48.902086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 644/1272] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:57:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:48.908003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3eea39-d58c-4ee4-bc5f-5143c3d9e055 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:48.996729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:49.005825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3eea39-d58c-4ee4-bc5f-5143c3d9e055 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:49.038583 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f8fcc-82ba-43c6-b465-6bef422f9001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:49.039256 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f8fcc-82ba-43c6-b465-6bef422f9001 None admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:49.039531 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f8fcc-82ba-43c6-b465-6bef422f9001 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:49.040068 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f8fcc-82ba-43c6-b465-6bef422f9001 None admin] PATH_INFO: `/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/6074872303d3468fb566378fa34e8bea/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:49.041648 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f8fcc-82ba-43c6-b465-6bef422f9001 None admin] RBAC: Authorizing `identity:create_grant(project_id=399f0095cc8b4c64893ccc75af13c52d, user_id=6074872303d3468fb566378fa34e8bea, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:49.043107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f8fcc-82ba-43c6-b465-6bef422f9001 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:49.160352 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 629/1273] 10.222.0.30 () {58 vars in 1439 bytes} [Thu Oct 6 07:57:48 2022] PUT /identity/v3/projects/399f0095cc8b4c64893ccc75af13c52d/users/6074872303d3468fb566378fa34e8bea/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 07:57:51.325507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01404db5-21d3-4911-9045-ca2d74c03e87 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:51.326078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01404db5-21d3-4911-9045-ca2d74c03e87 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:51.326078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01404db5-21d3-4911-9045-ca2d74c03e87 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:51.327560 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 645/1274] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:57:51.333566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19b3ab-3e70-40b6-942c-82adb1e41ecf None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:51.333982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19b3ab-3e70-40b6-942c-82adb1e41ecf None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:51.334137 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19b3ab-3e70-40b6-942c-82adb1e41ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:51.352232 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-0e19b3ab-3e70-40b6-942c-82adb1e41ecf None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:57:51.353423 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0e19b3ab-3e70-40b6-942c-82adb1e41ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:57:51.419516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 630/1275] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:57:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:57:51.708892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01404db5-21d3-4911-9045-ca2d74c03e87 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:51.814038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01404db5-21d3-4911-9045-ca2d74c03e87 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:51.830821 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:51.847271 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d764dd-b290-4daf-bb04-f79693aae8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:51.847881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d764dd-b290-4daf-bb04-f79693aae8dc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:51.848152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d764dd-b290-4daf-bb04-f79693aae8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:51.848413 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d764dd-b290-4daf-bb04-f79693aae8dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:51.858022 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d764dd-b290-4daf-bb04-f79693aae8dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 06 07:57:51.863311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d1d764dd-b290-4daf-bb04-f79693aae8dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:57:51.863311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:51.863311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:51.863311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:51.863311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:51.863311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:51.863311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:51.863311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:51.863311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:51.863311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:51.863311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:51.863311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:51.863311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 06 07:57:51.863311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 06 07:57:51.863311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 06 07:57:51.863311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 06 07:57:51.864414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 06 07:57:51.864414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 06 07:57:51.864414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 06 07:57:51.864414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 06 07:57:51.864414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:51.864414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:51.864414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:51.864414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:51.864414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:51.864414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:51.864414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:51.864414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:51.864414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:51.864414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:51.864414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:51.864414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:51.865494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:51.865494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:51.865494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:51.865494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:51.865494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 06 07:57:51.865494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 06 07:57:51.865494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 06 07:57:51.865494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 06 07:57:51.865494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 06 07:57:51.865494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 06 07:57:51.865494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 06 07:57:51.865494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:51.865494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 646/1276] 10.222.0.30 () {56 vars in 1096 bytes} [Thu Oct 6 07:57:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:51.870479 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19b3ab-3e70-40b6-942c-82adb1e41ecf None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:51.949930 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:51.967656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19b3ab-3e70-40b6-942c-82adb1e41ecf None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:52.001688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d57fba-503b-4f73-8c12-031335e3a2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:52.002281 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d57fba-503b-4f73-8c12-031335e3a2f9 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:52.002554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d57fba-503b-4f73-8c12-031335e3a2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:52.002904 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d57fba-503b-4f73-8c12-031335e3a2f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:52.003534 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d57fba-503b-4f73-8c12-031335e3a2f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:52.003853 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d57fba-503b-4f73-8c12-031335e3a2f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:52.005208 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d57fba-503b-4f73-8c12-031335e3a2f9 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:52.012911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 631/1277] 10.222.0.30 () {56 vars in 1099 bytes} [Thu Oct 6 07:57:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:52.019395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d764dd-b290-4daf-bb04-f79693aae8dc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:52.118009 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d764dd-b290-4daf-bb04-f79693aae8dc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:52.151734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d5cf-15fe-4398-b422-4039ab54709f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:52.152367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d5cf-15fe-4398-b422-4039ab54709f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:52.152997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d5cf-15fe-4398-b422-4039ab54709f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:52.152997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d5cf-15fe-4398-b422-4039ab54709f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:52.163623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98d5cf-15fe-4398-b422-4039ab54709f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:52.165094 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98d5cf-15fe-4398-b422-4039ab54709f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:52.179497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-cd98d5cf-15fe-4398-b422-4039ab54709f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:52.179497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:52.179497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:52.179497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:52.179497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:52.179497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:52.179497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:52.179497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:52.179497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:52.179497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:52.179497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:52.179497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:52.179497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:57:52.179497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:57:52.179497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:57:52.179497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:52.180494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:57:52.181533 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:57:52.181533 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:52.181533 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:52.181533 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 647/1278] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:52.186442 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d57fba-503b-4f73-8c12-031335e3a2f9 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:52.285921 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d57fba-503b-4f73-8c12-031335e3a2f9 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:52.320127 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c97e61-012a-4e9e-ae79-ee95d1f7deac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:52.320729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c97e61-012a-4e9e-ae79-ee95d1f7deac None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:52.321010 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c97e61-012a-4e9e-ae79-ee95d1f7deac None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:52.321280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c97e61-012a-4e9e-ae79-ee95d1f7deac None admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:52.321925 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c97e61-012a-4e9e-ae79-ee95d1f7deac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:52.322226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c97e61-012a-4e9e-ae79-ee95d1f7deac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:52.323640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c97e61-012a-4e9e-ae79-ee95d1f7deac None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:52.335433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 632/1279] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:52.341173 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d5cf-15fe-4398-b422-4039ab54709f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:52.372429 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:52.440409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d5cf-15fe-4398-b422-4039ab54709f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:52.474078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75146b-f3c5-4934-b972-e1fbf2f61fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:52.477676 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75146b-f3c5-4934-b972-e1fbf2f61fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:52.477676 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75146b-f3c5-4934-b972-e1fbf2f61fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:52.477676 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75146b-f3c5-4934-b972-e1fbf2f61fe5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:52.482026 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75146b-f3c5-4934-b972-e1fbf2f61fe5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:52.483488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75146b-f3c5-4934-b972-e1fbf2f61fe5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:52.495414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5b75146b-f3c5-4934-b972-e1fbf2f61fe5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 06 07:57:52.495414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:52.495414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:52.495414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:52.495414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:52.495414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:52.495414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:52.495414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:52.495414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:52.495414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:52.495414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:52.495414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:52.495414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 06 07:57:52.495414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 06 07:57:52.495414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 06 07:57:52.495414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:52.496511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:52.498111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:52.498111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:52.498111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 06 07:57:52.498111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 06 07:57:52.498111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 06 07:57:52.498111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 06 07:57:52.498111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 06 07:57:52.498111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 06 07:57:52.498111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 06 07:57:52.498111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:52.498111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 648/1280] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:52.503048 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c97e61-012a-4e9e-ae79-ee95d1f7deac None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:52.506786 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:52.601155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c97e61-012a-4e9e-ae79-ee95d1f7deac None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:52.635327 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f852a-a8f5-45c2-96f2-d759a41333f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:52.635913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f852a-a8f5-45c2-96f2-d759a41333f6 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:52.636186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f852a-a8f5-45c2-96f2-d759a41333f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:52.636454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f852a-a8f5-45c2-96f2-d759a41333f6 None admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:52.637161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f852a-a8f5-45c2-96f2-d759a41333f6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:52.637459 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f852a-a8f5-45c2-96f2-d759a41333f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:52.638863 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f852a-a8f5-45c2-96f2-d759a41333f6 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:52.655622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 633/1281] 10.222.0.30 () {56 vars in 1143 bytes} [Thu Oct 6 07:57:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:52.661946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75146b-f3c5-4934-b972-e1fbf2f61fe5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:52.752477 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:52.761701 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75146b-f3c5-4934-b972-e1fbf2f61fe5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:52.794948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053f156-5ab2-47d2-ba0f-a235c1e3b7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:52.795563 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053f156-5ab2-47d2-ba0f-a235c1e3b7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:52.795834 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053f156-5ab2-47d2-ba0f-a235c1e3b7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:52.796096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053f156-5ab2-47d2-ba0f-a235c1e3b7b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:52.860605 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053f156-5ab2-47d2-ba0f-a235c1e3b7b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:52.861980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053f156-5ab2-47d2-ba0f-a235c1e3b7b0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:52.955062 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 649/1282] 10.222.0.30 () {58 vars in 1282 bytes} [Thu Oct 6 07:57:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 06 07:57:52.960888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f852a-a8f5-45c2-96f2-d759a41333f6 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:53.005157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:53.022048 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:53.060597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f852a-a8f5-45c2-96f2-d759a41333f6 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:53.104954 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f992f40-72c9-46f3-be5d-e3260ce5c3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:53.105924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f992f40-72c9-46f3-be5d-e3260ce5c3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:53.106342 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f992f40-72c9-46f3-be5d-e3260ce5c3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:53.106743 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f992f40-72c9-46f3-be5d-e3260ce5c3d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:53.120834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f992f40-72c9-46f3-be5d-e3260ce5c3d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:53.123401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f992f40-72c9-46f3-be5d-e3260ce5c3d0 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:53.137755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-7f992f40-72c9-46f3-be5d-e3260ce5c3d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:53.137755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:53.137755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:53.137755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:53.137755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:53.137755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:53.137755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:53.137755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:53.137755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:53.137755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:53.137755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:53.137755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:53.137755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 06 07:57:53.137755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 06 07:57:53.137755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 06 07:57:53.137755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:53.138864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 06 07:57:53.139970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 06 07:57:53.139970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 06 07:57:53.139970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:57:53.139970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 634/1283] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 07:57:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:53.145361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053f156-5ab2-47d2-ba0f-a235c1e3b7b0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:53.243543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053f156-5ab2-47d2-ba0f-a235c1e3b7b0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:53.278560 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800509e-03fc-4768-842a-284175aa0310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:53.279182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800509e-03fc-4768-842a-284175aa0310 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:53.279455 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800509e-03fc-4768-842a-284175aa0310 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:53.279716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800509e-03fc-4768-842a-284175aa0310 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:53.280344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800509e-03fc-4768-842a-284175aa0310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:53.280637 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800509e-03fc-4768-842a-284175aa0310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:53.282006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800509e-03fc-4768-842a-284175aa0310 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:53.293148 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 650/1284] 10.222.0.30 () {56 vars in 1109 bytes} [Thu Oct 6 07:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:53.299261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f992f40-72c9-46f3-be5d-e3260ce5c3d0 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:53.401679 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f992f40-72c9-46f3-be5d-e3260ce5c3d0 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:53.416947 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:53.441167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655d227-c637-4ebe-b554-96da6afa69ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:53.441794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655d227-c637-4ebe-b554-96da6afa69ff None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:53.442071 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655d227-c637-4ebe-b554-96da6afa69ff None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:53.442342 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655d227-c637-4ebe-b554-96da6afa69ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:53.448748 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655d227-c637-4ebe-b554-96da6afa69ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:53.452348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655d227-c637-4ebe-b554-96da6afa69ff None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:53.461964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application [None req-c655d227-c637-4ebe-b554-96da6afa69ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:57:53.461964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:53.461964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:53.461964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:53.461964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:53.461964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:53.461964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:53.461964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:53.461964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:53.461964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:53.461964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:53.461964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:53.461964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 06 07:57:53.461964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 06 07:57:53.461964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:53.463044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:53.464205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 06 07:57:53.464205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 06 07:57:53.464205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 06 07:57:53.464205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 06 07:57:53.464205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 06 07:57:53.464205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 06 07:57:53.464205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 06 07:57:53.464205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: ERROR keystone.server.flask.application Oct 06 07:57:53.464205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 635/1285] 10.222.0.30 () {56 vars in 1111 bytes} [Thu Oct 6 07:57:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 06 07:57:53.468975 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800509e-03fc-4768-842a-284175aa0310 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:53.522912 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:53.574295 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800509e-03fc-4768-842a-284175aa0310 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:53.606685 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90662561-2d9d-4aa2-9b01-f61be16d47cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:53.607306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90662561-2d9d-4aa2-9b01-f61be16d47cf None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:53.607579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90662561-2d9d-4aa2-9b01-f61be16d47cf None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:53.607847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90662561-2d9d-4aa2-9b01-f61be16d47cf None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:53.608592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90662561-2d9d-4aa2-9b01-f61be16d47cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:53.608890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90662561-2d9d-4aa2-9b01-f61be16d47cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:53.610799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90662561-2d9d-4aa2-9b01-f61be16d47cf None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:53.622575 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 651/1286] 10.222.0.30 () {56 vars in 1148 bytes} [Thu Oct 6 07:57:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:53.629336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655d227-c637-4ebe-b554-96da6afa69ff None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:53.726687 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655d227-c637-4ebe-b554-96da6afa69ff None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:53.760447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15bc77-3387-4c10-93f2-6b98b8927c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:53.761027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15bc77-3387-4c10-93f2-6b98b8927c9c None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:53.761300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15bc77-3387-4c10-93f2-6b98b8927c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:53.761566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15bc77-3387-4c10-93f2-6b98b8927c9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:53.762278 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15bc77-3387-4c10-93f2-6b98b8927c9c None admin] RBAC: Adding query filter params (role.id=4c34f0f69c3a408fa32dbc52c64e7cfb, scope.project.id=399f0095cc8b4c64893ccc75af13c52d, user.id=6074872303d3468fb566378fa34e8bea) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:53.762583 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15bc77-3387-4c10-93f2-6b98b8927c9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:53.764037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15bc77-3387-4c10-93f2-6b98b8927c9c None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:53.769937 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 636/1287] 10.222.0.30 () {56 vars in 1374 bytes} [Thu Oct 6 07:57:53 2022] GET /identity/v3/role_assignments?user.id=6074872303d3468fb566378fa34e8bea&scope.project.id=399f0095cc8b4c64893ccc75af13c52d&role.id=4c34f0f69c3a408fa32dbc52c64e7cfb => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:56.017184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88982911-72ea-4e9b-b735-717140b52f8a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:56.017716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88982911-72ea-4e9b-b735-717140b52f8a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:56.017716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88982911-72ea-4e9b-b735-717140b52f8a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:56.019122 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 652/1288] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:57:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:57:56.024384 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc33726-0a75-4079-b596-518df8b5dd63 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:56.024823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc33726-0a75-4079-b596-518df8b5dd63 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:56.024924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc33726-0a75-4079-b596-518df8b5dd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:56.044209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-8fc33726-0a75-4079-b596-518df8b5dd63 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:57:56.045391 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-8fc33726-0a75-4079-b596-518df8b5dd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:57:56.074414 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:56.111125 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 637/1289] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:57:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:57:56.377949 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88982911-72ea-4e9b-b735-717140b52f8a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:56.479583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88982911-72ea-4e9b-b735-717140b52f8a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:56.490997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:56.514549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da053ba6-071a-45a6-bd0b-abe250b34a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:56.515299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da053ba6-071a-45a6-bd0b-abe250b34a49 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:56.515574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da053ba6-071a-45a6-bd0b-abe250b34a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:56.515841 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da053ba6-071a-45a6-bd0b-abe250b34a49 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:56.517182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da053ba6-071a-45a6-bd0b-abe250b34a49 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:56.518482 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da053ba6-071a-45a6-bd0b-abe250b34a49 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-da053ba6-071a-45a6-bd0b-abe250b34a49 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:57:56.526976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:57:56.528182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:57:56.528182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:57:56.528182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:57:56.528182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:57:56.528182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:57:56.528182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:57:56.528182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:57:56.528182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:57:56.528182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:57:56.528182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:57:56.528182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:57:56.528182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:57:56.528182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:57:56.528182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:57:56.528182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:57:56.528182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:57:56.529068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:57:56.529068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:57:56.529068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:57:56.529068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:57:56.529068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:57:56.529068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:57:56.529068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:57:56.529068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:57:56.529068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:57:56.529068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:57:56.529068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:57:56.529068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:57:56.529068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 07:57:56.529068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:57:56.529068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 653/1290] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Oct 6 07:57:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:57:56.534697 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc33726-0a75-4079-b596-518df8b5dd63 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:56.632281 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc33726-0a75-4079-b596-518df8b5dd63 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:56.664387 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881e1b0-ffea-44f2-95cd-9f5f0fbe199e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:56.664983 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881e1b0-ffea-44f2-95cd-9f5f0fbe199e None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:56.665258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881e1b0-ffea-44f2-95cd-9f5f0fbe199e None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:56.665597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881e1b0-ffea-44f2-95cd-9f5f0fbe199e None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:56.666224 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881e1b0-ffea-44f2-95cd-9f5f0fbe199e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:56.666516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881e1b0-ffea-44f2-95cd-9f5f0fbe199e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:56.667902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881e1b0-ffea-44f2-95cd-9f5f0fbe199e None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:56.673289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 638/1291] 10.222.0.30 () {56 vars in 1116 bytes} [Thu Oct 6 07:57:56 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:56.679252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da053ba6-071a-45a6-bd0b-abe250b34a49 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:56.778361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da053ba6-071a-45a6-bd0b-abe250b34a49 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:56.810867 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd262f8a-fd31-4b14-b3c1-193bc9baaeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:56.811482 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd262f8a-fd31-4b14-b3c1-193bc9baaeb2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:56.811755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd262f8a-fd31-4b14-b3c1-193bc9baaeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:56.812024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd262f8a-fd31-4b14-b3c1-193bc9baaeb2 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:56.812645 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd262f8a-fd31-4b14-b3c1-193bc9baaeb2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:57:56.813710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd262f8a-fd31-4b14-b3c1-193bc9baaeb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:56.815096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd262f8a-fd31-4b14-b3c1-193bc9baaeb2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:56.819357 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 654/1292] 10.222.0.30 () {56 vars in 1116 bytes} [Thu Oct 6 07:57:56 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:57:58.985369 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232455d2-b111-4344-a2ee-e2039fa12f41 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:58.985947 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232455d2-b111-4344-a2ee-e2039fa12f41 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:58.986132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232455d2-b111-4344-a2ee-e2039fa12f41 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:58.987523 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 639/1293] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:57:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:57:58.992908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f576c3e-c706-464f-b2a2-c518fae82cf5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:58.993313 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f576c3e-c706-464f-b2a2-c518fae82cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:58.993498 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f576c3e-c706-464f-b2a2-c518fae82cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:59.012122 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4f576c3e-c706-464f-b2a2-c518fae82cf5 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:57:59.013303 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4f576c3e-c706-464f-b2a2-c518fae82cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:57:59.041748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:59.079135 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 655/1294] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:57:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:57:59.356839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232455d2-b111-4344-a2ee-e2039fa12f41 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:57:59.392381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:57:59.461388 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232455d2-b111-4344-a2ee-e2039fa12f41 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:57:59.501449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e31e89-eac5-40e5-ad67-9aeedd9b3088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:57:59.502046 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e31e89-eac5-40e5-ad67-9aeedd9b3088 None admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:57:59.502326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e31e89-eac5-40e5-ad67-9aeedd9b3088 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:57:59.502595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e31e89-eac5-40e5-ad67-9aeedd9b3088 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:57:59.503282 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e31e89-eac5-40e5-ad67-9aeedd9b3088 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:57:59.504656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e31e89-eac5-40e5-ad67-9aeedd9b3088 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:57:59.559222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 640/1295] 10.222.0.30 () {60 vars in 1141 bytes} [Thu Oct 6 07:57:59 2022] POST /identity/v3/services => generated 246 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:58:01.742094 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6d9f6-7ac6-4ec6-a240-afeeae0dad42 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:58:01.742628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6d9f6-7ac6-4ec6-a240-afeeae0dad42 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:58:01.742752 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6d9f6-7ac6-4ec6-a240-afeeae0dad42 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:58:01.744076 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 656/1296] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:58:01.749687 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684ac6d-5416-4ca7-9c0f-d82bea02cff8 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:58:01.750657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684ac6d-5416-4ca7-9c0f-d82bea02cff8 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:58:01.750802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684ac6d-5416-4ca7-9c0f-d82bea02cff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:58:01.769602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-f684ac6d-5416-4ca7-9c0f-d82bea02cff8 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:58:01.770793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f684ac6d-5416-4ca7-9c0f-d82bea02cff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:58:01.841601 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 641/1297] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:58:01 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 06 07:58:02.109342 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6d9f6-7ac6-4ec6-a240-afeeae0dad42 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:58:02.210392 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6d9f6-7ac6-4ec6-a240-afeeae0dad42 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:58:02.246400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954038ab-da44-4c76-8839-093c40f4b93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:58:02.247315 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954038ab-da44-4c76-8839-093c40f4b93f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:58:02.247582 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954038ab-da44-4c76-8839-093c40f4b93f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:58:02.247841 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954038ab-da44-4c76-8839-093c40f4b93f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:58:02.248420 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954038ab-da44-4c76-8839-093c40f4b93f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:58:02.249779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954038ab-da44-4c76-8839-093c40f4b93f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-954038ab-da44-4c76-8839-093c40f4b93f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:58:02.258774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:58:02.263152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:58:02.263152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:58:02.263152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:58:02.263152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:58:02.263152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:58:02.263152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:58:02.263152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:58:02.263152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:58:02.263152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:58:02.263152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:58:02.263152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:58:02.263152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:58:02.263152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:58:02.263152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:58:02.263152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:58:02.263152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:58:02.264252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:58:02.264252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:58:02.264252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:58:02.264252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:58:02.264252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:58:02.264252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:58:02.264252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:58:02.264252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:58:02.264252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:58:02.264252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:58:02.264252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:58:02.264252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:58:02.264252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 07:58:02.264252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:58:02.264252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 657/1298] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Oct 6 07:58:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:58:02.267081 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684ac6d-5416-4ca7-9c0f-d82bea02cff8 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:58:02.285801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:58:02.367137 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684ac6d-5416-4ca7-9c0f-d82bea02cff8 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:58:02.401711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108c57b-79ac-4089-88f2-b37b3f38e3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:58:02.402308 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108c57b-79ac-4089-88f2-b37b3f38e3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:58:02.402663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108c57b-79ac-4089-88f2-b37b3f38e3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:58:02.402961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108c57b-79ac-4089-88f2-b37b3f38e3f4 None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:58:02.403582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108c57b-79ac-4089-88f2-b37b3f38e3f4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:58:02.403883 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108c57b-79ac-4089-88f2-b37b3f38e3f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:58:02.405250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108c57b-79ac-4089-88f2-b37b3f38e3f4 None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:58:02.410791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 642/1299] 10.222.0.30 () {56 vars in 1116 bytes} [Thu Oct 6 07:58:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:58:02.416809 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954038ab-da44-4c76-8839-093c40f4b93f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:58:02.447832 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:58:02.514874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954038ab-da44-4c76-8839-093c40f4b93f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:58:02.548132 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40524b36-c155-4012-b08a-eb20db502a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:58:02.548721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40524b36-c155-4012-b08a-eb20db502a6a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:58:02.548996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40524b36-c155-4012-b08a-eb20db502a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:58:02.549263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40524b36-c155-4012-b08a-eb20db502a6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:58:02.549949 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40524b36-c155-4012-b08a-eb20db502a6a None admin] RBAC: Adding query filter params (interface=public, service_id=dc4f4496037d40c29d1099e65d1cab53, region_id=RegionOne) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:58:02.550257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40524b36-c155-4012-b08a-eb20db502a6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:58:02.551668 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40524b36-c155-4012-b08a-eb20db502a6a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:58:02.556571 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 658/1300] 10.222.0.30 () {56 vars in 1251 bytes} [Thu Oct 6 07:58:02 2022] GET /identity/v3/endpoints?service_id=dc4f4496037d40c29d1099e65d1cab53&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 07:58:02.562306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108c57b-79ac-4089-88f2-b37b3f38e3f4 None admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:58:02.642174 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:58:02.659126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108c57b-79ac-4089-88f2-b37b3f38e3f4 None admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:58:02.693412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55911cd-61e7-4a17-aa5e-cf3c2f2a3aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:58:02.694047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55911cd-61e7-4a17-aa5e-cf3c2f2a3aec None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:58:02.694321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55911cd-61e7-4a17-aa5e-cf3c2f2a3aec None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:58:02.694583 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55911cd-61e7-4a17-aa5e-cf3c2f2a3aec None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:58:02.695207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55911cd-61e7-4a17-aa5e-cf3c2f2a3aec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:58:02.696562 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55911cd-61e7-4a17-aa5e-cf3c2f2a3aec None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:58:02.703349 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 643/1301] 10.222.0.30 () {56 vars in 1087 bytes} [Thu Oct 6 07:58:02 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 07:58:04.860941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a73ed3-7284-4baf-8522-bc8e372b2601 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:58:04.861557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a73ed3-7284-4baf-8522-bc8e372b2601 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:58:04.861557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a73ed3-7284-4baf-8522-bc8e372b2601 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:58:04.862856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 659/1302] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 07:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 07:58:04.868181 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c2a5e-ab22-40fc-93a3-d99fe6fd635e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:58:04.868593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c2a5e-ab22-40fc-93a3-d99fe6fd635e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:58:04.868734 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c2a5e-ab22-40fc-93a3-d99fe6fd635e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:58:04.886796 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-ce6c2a5e-ab22-40fc-93a3-d99fe6fd635e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 07:58:04.888012 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-ce6c2a5e-ab22-40fc-93a3-d99fe6fd635e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 07:58:04.951601 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 644/1303] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 07:58:04 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 06 07:58:05.218275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a73ed3-7284-4baf-8522-bc8e372b2601 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:58:05.312972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:58:05.319677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a73ed3-7284-4baf-8522-bc8e372b2601 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:58:05.352999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7ff67-6ed1-43ab-9ae9-36eb849a7066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:58:05.353615 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7ff67-6ed1-43ab-9ae9-36eb849a7066 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:58:05.353889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7ff67-6ed1-43ab-9ae9-36eb849a7066 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:58:05.354157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7ff67-6ed1-43ab-9ae9-36eb849a7066 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:58:05.354732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7ff67-6ed1-43ab-9ae9-36eb849a7066 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:58:05.356117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7ff67-6ed1-43ab-9ae9-36eb849a7066 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-4cd7ff67-6ed1-43ab-9ae9-36eb849a7066 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 06 07:58:05.364699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 06 07:58:05.368055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 06 07:58:05.368055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 06 07:58:05.368055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 06 07:58:05.368055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 06 07:58:05.368055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 06 07:58:05.368055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 06 07:58:05.368055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 06 07:58:05.368055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 06 07:58:05.368055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 06 07:58:05.368055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 06 07:58:05.368055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 06 07:58:05.368055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 06 07:58:05.368055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 06 07:58:05.368055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 06 07:58:05.368055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 06 07:58:05.368055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 06 07:58:05.369209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 06 07:58:05.369209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 06 07:58:05.369209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 06 07:58:05.369209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 06 07:58:05.369209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 06 07:58:05.369209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 06 07:58:05.369209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 06 07:58:05.369209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 06 07:58:05.369209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 06 07:58:05.369209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 06 07:58:05.369209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 06 07:58:05.369209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 06 07:58:05.369209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 06 07:58:05.369209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 06 07:58:05.369209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 660/1304] 10.222.0.30 () {56 vars in 1117 bytes} [Thu Oct 6 07:58:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 06 07:58:05.372087 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c2a5e-ab22-40fc-93a3-d99fe6fd635e None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:58:05.447939 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:58:05.470934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c2a5e-ab22-40fc-93a3-d99fe6fd635e None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:58:05.504851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a1492-3836-4ad1-8c5f-955025ffc63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:58:05.505482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a1492-3836-4ad1-8c5f-955025ffc63f None admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:58:05.505755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a1492-3836-4ad1-8c5f-955025ffc63f None admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:58:05.506020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a1492-3836-4ad1-8c5f-955025ffc63f None admin] PATH_INFO: `/v3/services` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:58:05.506637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a1492-3836-4ad1-8c5f-955025ffc63f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 07:58:05.506924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a1492-3836-4ad1-8c5f-955025ffc63f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:58:05.508311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a1492-3836-4ad1-8c5f-955025ffc63f None admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:58:05.513350 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 645/1305] 10.222.0.30 () {56 vars in 1116 bytes} [Thu Oct 6 07:58:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 07:58:05.520338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7ff67-6ed1-43ab-9ae9-36eb849a7066 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 07:58:05.626183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7ff67-6ed1-43ab-9ae9-36eb849a7066 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 07:58:05.658324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94725c1-1d0b-4c78-951c-d716b78cd31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 07:58:05.658909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94725c1-1d0b-4c78-951c-d716b78cd31f None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 07:58:05.659209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94725c1-1d0b-4c78-951c-d716b78cd31f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 07:58:05.659472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94725c1-1d0b-4c78-951c-d716b78cd31f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 07:58:05.660231 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94725c1-1d0b-4c78-951c-d716b78cd31f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 07:58:05.661577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94725c1-1d0b-4c78-951c-d716b78cd31f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 07:58:05.764417 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 661/1306] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 07:58:05 2022] POST /identity/v3/endpoints => generated 331 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 07:58:15.558401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=30 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 07:58:15.759845 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:04:45.955465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e4e44-374e-45f2-b71a-a46198a2aed7 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:04:45.956638 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e4e44-374e-45f2-b71a-a46198a2aed7 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:04:45.957223 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e4e44-374e-45f2-b71a-a46198a2aed7 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:04:45.965548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc864fd-d07c-4d85-8afe-3366d8f72759 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:04:45.967121 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc864fd-d07c-4d85-8afe-3366d8f72759 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:04:45.968213 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc864fd-d07c-4d85-8afe-3366d8f72759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:04:45.973768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 646/1307] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:04:45 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:04:46.124706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cfc864fd-d07c-4d85-8afe-3366d8f72759 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:04:46.125933 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cfc864fd-d07c-4d85-8afe-3366d8f72759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:04:46.247030 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 662/1308] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:04:46.536174 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be38a3c-c142-46c0-bdc3-e8bb9760fe52 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:04:46.536817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be38a3c-c142-46c0-bdc3-e8bb9760fe52 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:04:46.537249 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be38a3c-c142-46c0-bdc3-e8bb9760fe52 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:04:46.538781 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 647/1309] 10.222.0.30 () {54 vars in 964 bytes} [Thu Oct 6 08:04:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:04:46.545905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e2919-41c2-48cb-870a-6d2fd7c65c6b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:04:46.546204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e2919-41c2-48cb-870a-6d2fd7c65c6b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:04:46.546773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e2919-41c2-48cb-870a-6d2fd7c65c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:04:46.640204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-372e2919-41c2-48cb-870a-6d2fd7c65c6b None None] MFA Rules not processed for user `bf6798f89acf44e897c481924edd9047`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:04:46.641483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-372e2919-41c2-48cb-870a-6d2fd7c65c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:04:46.664760 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:04:46.725830 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 663/1310] 10.222.0.30 () {58 vars in 1061 bytes} [Thu Oct 6 08:04:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:04:46.734999 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be38a3c-c142-46c0-bdc3-e8bb9760fe52 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:04:46.737132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5be38a3c-c142-46c0-bdc3-e8bb9760fe52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:04:46.849635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be38a3c-c142-46c0-bdc3-e8bb9760fe52 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:04:46.927143 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f211d85-dbcd-4bd8-94ba-63d5ff498895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:04:46.927752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f211d85-dbcd-4bd8-94ba-63d5ff498895 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:04:46.928028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f211d85-dbcd-4bd8-94ba-63d5ff498895 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:04:46.928294 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f211d85-dbcd-4bd8-94ba-63d5ff498895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:04:46.929928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-2f211d85-dbcd-4bd8-94ba-63d5ff498895 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:04:46.988282 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:04:47.047556 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f211d85-dbcd-4bd8-94ba-63d5ff498895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:04:47.049675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f211d85-dbcd-4bd8-94ba-63d5ff498895 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:04:47.069736 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 648/1311] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:04:49.642127 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da230d4-e31f-4a0b-bfc4-db876d2db328 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:04:49.642648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da230d4-e31f-4a0b-bfc4-db876d2db328 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:04:49.642648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da230d4-e31f-4a0b-bfc4-db876d2db328 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:04:49.643972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 664/1312] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:04:49.649108 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde2ac9-9e84-4720-a8b2-8780add7f4a0 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:04:49.649477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde2ac9-9e84-4720-a8b2-8780add7f4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:04:49.649659 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde2ac9-9e84-4720-a8b2-8780add7f4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:04:49.720066 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-fbde2ac9-9e84-4720-a8b2-8780add7f4a0 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:04:49.721271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-fbde2ac9-9e84-4720-a8b2-8780add7f4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:04:49.814911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 649/1313] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:04:50.097212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da230d4-e31f-4a0b-bfc4-db876d2db328 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:04:50.098793 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3da230d4-e31f-4a0b-bfc4-db876d2db328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:04:50.207411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da230d4-e31f-4a0b-bfc4-db876d2db328 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:04:50.278785 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:04:50.281195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c4f90-230a-4808-b2c4-d7768a6bc3de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:04:50.281797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4f90-230a-4808-b2c4-d7768a6bc3de service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:04:50.282072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4f90-230a-4808-b2c4-d7768a6bc3de service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:04:50.282339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4f90-230a-4808-b2c4-d7768a6bc3de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:04:50.283977 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-292c4f90-230a-4808-b2c4-d7768a6bc3de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:04:50.392028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c4f90-230a-4808-b2c4-d7768a6bc3de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:04:50.393457 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c4f90-230a-4808-b2c4-d7768a6bc3de service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:04:50.411914 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 665/1314] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:04:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:04:53.115629 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00dab2-4e91-466f-b64c-06c06be5daed None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:04:53.116160 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00dab2-4e91-466f-b64c-06c06be5daed None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:04:53.116160 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00dab2-4e91-466f-b64c-06c06be5daed None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:04:53.117511 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 650/1315] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:04:53.123084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ddba6-8c88-4148-9080-8b0aa6650f16 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:04:53.123495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ddba6-8c88-4148-9080-8b0aa6650f16 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:04:53.123633 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ddba6-8c88-4148-9080-8b0aa6650f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:04:53.189973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-656ddba6-8c88-4148-9080-8b0aa6650f16 None None] MFA Rules not processed for user `ffb9e4ff6b76404b9d9afd02933c035e`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:04:53.191219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-656ddba6-8c88-4148-9080-8b0aa6650f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:04:53.243151 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:04:53.286506 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 666/1316] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:04:53 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:04:53.601250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f7fe6-b598-4ce0-85ad-5b44278e0fb5 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:04:53.601765 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f7fe6-b598-4ce0-85ad-5b44278e0fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:04:53.602034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f7fe6-b598-4ce0-85ad-5b44278e0fb5 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:04:53.603426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 651/1317] 10.222.0.30 () {54 vars in 964 bytes} [Thu Oct 6 08:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:04:53.609682 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d4000-2527-484f-9645-edb006e40529 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:04:53.609964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d4000-2527-484f-9645-edb006e40529 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:04:53.610237 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d4000-2527-484f-9645-edb006e40529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:04:53.685476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ae7d4000-2527-484f-9645-edb006e40529 None None] MFA Rules not processed for user `bf6798f89acf44e897c481924edd9047`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:04:53.686676 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ae7d4000-2527-484f-9645-edb006e40529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:04:53.760383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 667/1318] 10.222.0.30 () {58 vars in 1061 bytes} [Thu Oct 6 08:04:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:04:53.769474 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f7fe6-b598-4ce0-85ad-5b44278e0fb5 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:04:53.771037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d99f7fe6-b598-4ce0-85ad-5b44278e0fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:04:53.780533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:04:53.874187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f7fe6-b598-4ce0-85ad-5b44278e0fb5 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:04:53.946519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6592c-5c0e-4463-ae41-f47809f06291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:04:53.947147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6592c-5c0e-4463-ae41-f47809f06291 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:04:53.947417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6592c-5c0e-4463-ae41-f47809f06291 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:04:53.947679 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6592c-5c0e-4463-ae41-f47809f06291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:04:53.949259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e5a6592c-5c0e-4463-ae41-f47809f06291 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:04:54.008692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:04:54.053944 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6592c-5c0e-4463-ae41-f47809f06291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:04:54.055404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6592c-5c0e-4463-ae41-f47809f06291 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:04:54.072614 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 652/1319] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:04:53 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:04:57.366445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0565a-454f-48f2-8414-2ef6d2489237 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:04:57.366965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0565a-454f-48f2-8414-2ef6d2489237 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:04:57.367060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0565a-454f-48f2-8414-2ef6d2489237 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:04:57.368362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 668/1320] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:04:57.374091 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4e4f0-ca49-4a8e-8648-93686edfd2dd None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:04:57.374466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4e4f0-ca49-4a8e-8648-93686edfd2dd None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:04:57.374628 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4e4f0-ca49-4a8e-8648-93686edfd2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:04:57.439615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-0ba4e4f0-ca49-4a8e-8648-93686edfd2dd None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:04:57.440765 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0ba4e4f0-ca49-4a8e-8648-93686edfd2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:04:57.534605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 653/1321] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:04:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:04:57.820393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0565a-454f-48f2-8414-2ef6d2489237 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:04:57.821964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a6b0565a-454f-48f2-8414-2ef6d2489237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:04:57.872586 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:04:57.925921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0565a-454f-48f2-8414-2ef6d2489237 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:04:57.997314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ed03a-43fe-4006-a01f-37c6e77dc347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:04:57.997898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ed03a-43fe-4006-a01f-37c6e77dc347 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:04:57.998170 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ed03a-43fe-4006-a01f-37c6e77dc347 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:04:57.998436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ed03a-43fe-4006-a01f-37c6e77dc347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:04:58.000045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bf6ed03a-43fe-4006-a01f-37c6e77dc347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:04:58.103955 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:04:58.120029 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ed03a-43fe-4006-a01f-37c6e77dc347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:04:58.143002 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ed03a-43fe-4006-a01f-37c6e77dc347 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:04:58.143568 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 669/1322] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:04:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:05:01.406048 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926344d-ac89-4531-9859-56613fd274b6 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:01.406577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926344d-ac89-4531-9859-56613fd274b6 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:01.406577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926344d-ac89-4531-9859-56613fd274b6 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:01.407876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 654/1323] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:01.412971 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565541e-dc06-43ad-92d3-c8a399af39c9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:01.413394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565541e-dc06-43ad-92d3-c8a399af39c9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:01.413527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565541e-dc06-43ad-92d3-c8a399af39c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:01.479648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d565541e-dc06-43ad-92d3-c8a399af39c9 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:01.481003 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d565541e-dc06-43ad-92d3-c8a399af39c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:01.581812 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 670/1324] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:01.862466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6926344d-ac89-4531-9859-56613fd274b6 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:05:01.864199 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6926344d-ac89-4531-9859-56613fd274b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:01.884157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:01.968376 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6926344d-ac89-4531-9859-56613fd274b6 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:05:02.043730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f592678-deb6-4cb4-bafe-e8d96a398326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:05:02.044769 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f592678-deb6-4cb4-bafe-e8d96a398326 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:02.045045 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f592678-deb6-4cb4-bafe-e8d96a398326 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:02.045309 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f592678-deb6-4cb4-bafe-e8d96a398326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:02.046896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-8f592678-deb6-4cb4-bafe-e8d96a398326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:02.117765 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:02.157272 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f592678-deb6-4cb4-bafe-e8d96a398326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:05:02.159706 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f592678-deb6-4cb4-bafe-e8d96a398326 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:05:02.178019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 655/1325] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:05:04.425622 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d8f18-d894-4885-af48-3716ee67b64f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:04.426152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d8f18-d894-4885-af48-3716ee67b64f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:04.426152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d8f18-d894-4885-af48-3716ee67b64f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:04.427480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 671/1326] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:04.432719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619ff6e-b9ae-4c49-96fb-036c2604c7a4 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:04.433887 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619ff6e-b9ae-4c49-96fb-036c2604c7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:04.434051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619ff6e-b9ae-4c49-96fb-036c2604c7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:04.510162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-2619ff6e-b9ae-4c49-96fb-036c2604c7a4 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:04.511594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-2619ff6e-b9ae-4c49-96fb-036c2604c7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:04.608358 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 656/1327] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:04.903627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d8f18-d894-4885-af48-3716ee67b64f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:05:04.905195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-392d8f18-d894-4885-af48-3716ee67b64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:04.966712 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:05.009504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d8f18-d894-4885-af48-3716ee67b64f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:05:05.082786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8265fd-f3f3-43e5-bd71-f6a7b9e7e7ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:05:05.083418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8265fd-f3f3-43e5-bd71-f6a7b9e7e7ed service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:05.083694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8265fd-f3f3-43e5-bd71-f6a7b9e7e7ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:05.083965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8265fd-f3f3-43e5-bd71-f6a7b9e7e7ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:05.085545 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1f8265fd-f3f3-43e5-bd71-f6a7b9e7e7ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:05.195953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:05.198285 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8265fd-f3f3-43e5-bd71-f6a7b9e7e7ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:05:05.200325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8265fd-f3f3-43e5-bd71-f6a7b9e7e7ed service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:05:05.218496 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 672/1328] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:05:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:05:07.457914 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c82f7f-8c34-4a48-902a-561bcf120f0d None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:07.458457 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c82f7f-8c34-4a48-902a-561bcf120f0d None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:07.458457 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c82f7f-8c34-4a48-902a-561bcf120f0d None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:07.459855 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 657/1329] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:07.465906 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce43f6-c7df-4c2b-862b-ea3acbe777be None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:07.466360 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce43f6-c7df-4c2b-862b-ea3acbe777be None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:07.466644 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce43f6-c7df-4c2b-862b-ea3acbe777be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:07.535653 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-abce43f6-c7df-4c2b-862b-ea3acbe777be None None] MFA Rules not processed for user `ffb9e4ff6b76404b9d9afd02933c035e`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:07.536898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-abce43f6-c7df-4c2b-862b-ea3acbe777be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:07.638483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 673/1330] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:05:07 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:07.938328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c82f7f-8c34-4a48-902a-561bcf120f0d None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:05:07.939962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b3c82f7f-8c34-4a48-902a-561bcf120f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:07.972622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:08.049883 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c82f7f-8c34-4a48-902a-561bcf120f0d None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:05:08.130062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3da68-a4ce-43aa-8fa8-524369bc93bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:05:08.130664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3da68-a4ce-43aa-8fa8-524369bc93bd service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:08.130951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3da68-a4ce-43aa-8fa8-524369bc93bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:08.131226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3da68-a4ce-43aa-8fa8-524369bc93bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:08.132814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-73d3da68-a4ce-43aa-8fa8-524369bc93bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:08.217535 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:08.250067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d3da68-a4ce-43aa-8fa8-524369bc93bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:05:08.251846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d3da68-a4ce-43aa-8fa8-524369bc93bd service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:05:08.270421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 658/1331] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:05:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:05:11.246826 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d7d91-d1db-4f5f-8fd5-fe41e4907e74 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:11.247401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d7d91-d1db-4f5f-8fd5-fe41e4907e74 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:11.247401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d7d91-d1db-4f5f-8fd5-fe41e4907e74 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:11.248758 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 674/1332] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:11.254649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02089070-367b-4b57-b6fd-6a1822479410 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:11.255080 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02089070-367b-4b57-b6fd-6a1822479410 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:11.255221 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02089070-367b-4b57-b6fd-6a1822479410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:11.319807 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-02089070-367b-4b57-b6fd-6a1822479410 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:11.320993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-02089070-367b-4b57-b6fd-6a1822479410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:11.415708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 659/1333] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:11.698108 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d7d91-d1db-4f5f-8fd5-fe41e4907e74 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:05:11.700176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-921d7d91-d1db-4f5f-8fd5-fe41e4907e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:11.768553 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:11.801874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d7d91-d1db-4f5f-8fd5-fe41e4907e74 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:05:11.876270 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a8bfb-fefd-4c72-91a6-e5add8e8317b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:05:11.876872 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a8bfb-fefd-4c72-91a6-e5add8e8317b service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:11.877145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a8bfb-fefd-4c72-91a6-e5add8e8317b service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:11.877407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a8bfb-fefd-4c72-91a6-e5add8e8317b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:11.879266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7d4a8bfb-fefd-4c72-91a6-e5add8e8317b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:11.988098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a8bfb-fefd-4c72-91a6-e5add8e8317b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:05:11.989559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a8bfb-fefd-4c72-91a6-e5add8e8317b service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:05:11.999856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:12.124819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 675/1334] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:05:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:05:14.420474 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6773eb0-8d19-4508-958b-ad534cce22f9 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:14.420993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6773eb0-8d19-4508-958b-ad534cce22f9 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:14.420993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6773eb0-8d19-4508-958b-ad534cce22f9 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:14.422289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 660/1335] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:14.427526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb1cc5-d0c5-4304-9b81-697033721113 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:14.427964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb1cc5-d0c5-4304-9b81-697033721113 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:14.428069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb1cc5-d0c5-4304-9b81-697033721113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:14.493314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b0cb1cc5-d0c5-4304-9b81-697033721113 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:14.494517 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b0cb1cc5-d0c5-4304-9b81-697033721113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:14.590376 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 676/1336] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:14.883359 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6773eb0-8d19-4508-958b-ad534cce22f9 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:05:14.884918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a6773eb0-8d19-4508-958b-ad534cce22f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:14.926123 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:14.990018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6773eb0-8d19-4508-958b-ad534cce22f9 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:05:15.063455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34a058-82af-49fa-aa62-62d1d0a78733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:05:15.064058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34a058-82af-49fa-aa62-62d1d0a78733 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:15.064335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34a058-82af-49fa-aa62-62d1d0a78733 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:15.064602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34a058-82af-49fa-aa62-62d1d0a78733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:15.066198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-ba34a058-82af-49fa-aa62-62d1d0a78733 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:15.149442 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:15.176537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34a058-82af-49fa-aa62-62d1d0a78733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:05:15.177990 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34a058-82af-49fa-aa62-62d1d0a78733 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:05:15.197603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 661/1337] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:05:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:05:17.919148 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab37a6-f765-4137-a015-7af0d6346fa1 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:17.919655 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab37a6-f765-4137-a015-7af0d6346fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:17.919655 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab37a6-f765-4137-a015-7af0d6346fa1 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:17.920943 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 677/1338] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:17.926075 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c500ac4-a596-4ee6-816b-8247a85efb2e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:17.926516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c500ac4-a596-4ee6-816b-8247a85efb2e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:17.926623 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c500ac4-a596-4ee6-816b-8247a85efb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:17.992152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-4c500ac4-a596-4ee6-816b-8247a85efb2e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:17.993345 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4c500ac4-a596-4ee6-816b-8247a85efb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:18.087495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 662/1339] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:05:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:18.365242 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab37a6-f765-4137-a015-7af0d6346fa1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:05:18.366747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e4ab37a6-f765-4137-a015-7af0d6346fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:18.441701 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:18.467806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab37a6-f765-4137-a015-7af0d6346fa1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:05:18.541276 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229fb7e-a5dd-4552-ba63-7464a072d6de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:05:18.542438 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229fb7e-a5dd-4552-ba63-7464a072d6de service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:18.542932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229fb7e-a5dd-4552-ba63-7464a072d6de service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:18.543065 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229fb7e-a5dd-4552-ba63-7464a072d6de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:18.551391 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6229fb7e-a5dd-4552-ba63-7464a072d6de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:18.658624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6229fb7e-a5dd-4552-ba63-7464a072d6de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:05:18.660040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6229fb7e-a5dd-4552-ba63-7464a072d6de service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:05:18.678496 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 678/1340] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:05:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:05:23.062232 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de2abb-651a-46b5-9221-3aecbde9e673 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:23.062788 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de2abb-651a-46b5-9221-3aecbde9e673 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:23.062788 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de2abb-651a-46b5-9221-3aecbde9e673 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:23.064180 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 663/1341] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:23.069536 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f20e5-f80d-41ff-8c28-9b4eb82ec80f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:23.069940 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f20e5-f80d-41ff-8c28-9b4eb82ec80f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:23.070212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f20e5-f80d-41ff-8c28-9b4eb82ec80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:23.076493 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:23.139073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4d7f20e5-f80d-41ff-8c28-9b4eb82ec80f None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:23.140230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4d7f20e5-f80d-41ff-8c28-9b4eb82ec80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:23.238397 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 679/1342] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:23.529552 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de2abb-651a-46b5-9221-3aecbde9e673 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:05:23.531301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a9de2abb-651a-46b5-9221-3aecbde9e673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:23.586894 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:23.643321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de2abb-651a-46b5-9221-3aecbde9e673 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:05:23.716528 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896f380-72bc-4527-b407-2eae0b00b41f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:05:23.717133 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896f380-72bc-4527-b407-2eae0b00b41f service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:23.717411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896f380-72bc-4527-b407-2eae0b00b41f service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:23.717689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896f380-72bc-4527-b407-2eae0b00b41f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:23.719321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1896f380-72bc-4527-b407-2eae0b00b41f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:23.812406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:23.828420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896f380-72bc-4527-b407-2eae0b00b41f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:05:23.829853 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896f380-72bc-4527-b407-2eae0b00b41f service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:05:23.847674 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 664/1343] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:05:27.250474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a6a05-4b8f-4104-a285-dee7fe1ea433 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:27.251084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a6a05-4b8f-4104-a285-dee7fe1ea433 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:27.251084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a6a05-4b8f-4104-a285-dee7fe1ea433 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:27.252386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 680/1344] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:27.257876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fb507-2e17-40a5-b39c-ce17a7e9a6de None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:27.258290 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fb507-2e17-40a5-b39c-ce17a7e9a6de None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:27.258417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fb507-2e17-40a5-b39c-ce17a7e9a6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:27.327629 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-e32fb507-2e17-40a5-b39c-ce17a7e9a6de None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:27.328838 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e32fb507-2e17-40a5-b39c-ce17a7e9a6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:27.428044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 665/1345] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:05:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:27.715289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a6a05-4b8f-4104-a285-dee7fe1ea433 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:05:27.716858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c55a6a05-4b8f-4104-a285-dee7fe1ea433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:27.821107 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:27.843099 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a6a05-4b8f-4104-a285-dee7fe1ea433 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:05:27.924476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bad9a-f2e0-41fb-a769-2d62882f7abc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:05:27.925087 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bad9a-f2e0-41fb-a769-2d62882f7abc service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:27.925359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bad9a-f2e0-41fb-a769-2d62882f7abc service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:27.925625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bad9a-f2e0-41fb-a769-2d62882f7abc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:27.927287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-693bad9a-f2e0-41fb-a769-2d62882f7abc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:28.042853 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bad9a-f2e0-41fb-a769-2d62882f7abc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:05:28.044481 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bad9a-f2e0-41fb-a769-2d62882f7abc service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:05:28.064076 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 681/1346] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:05:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:05:32.394549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79186e2-7d2f-45ab-b22d-665f079dd0b6 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:32.395157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79186e2-7d2f-45ab-b22d-665f079dd0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:32.395511 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79186e2-7d2f-45ab-b22d-665f079dd0b6 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:32.396585 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 666/1347] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:32.402908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce0247-b7e0-45c7-aa98-f3c10f61c216 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:32.403459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce0247-b7e0-45c7-aa98-f3c10f61c216 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:32.403557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce0247-b7e0-45c7-aa98-f3c10f61c216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:32.419802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:32.473957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-22ce0247-b7e0-45c7-aa98-f3c10f61c216 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:32.475354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-22ce0247-b7e0-45c7-aa98-f3c10f61c216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:32.576913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 682/1348] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:32.863248 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79186e2-7d2f-45ab-b22d-665f079dd0b6 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:05:32.864792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d79186e2-7d2f-45ab-b22d-665f079dd0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:32.926364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:32.975148 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79186e2-7d2f-45ab-b22d-665f079dd0b6 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:05:33.048853 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6a4fb-17f3-456c-874b-6aaee269ace0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:05:33.049462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6a4fb-17f3-456c-874b-6aaee269ace0 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:33.049739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6a4fb-17f3-456c-874b-6aaee269ace0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:33.050205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6a4fb-17f3-456c-874b-6aaee269ace0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:33.051842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a9e6a4fb-17f3-456c-874b-6aaee269ace0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:33.156782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:33.160289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6a4fb-17f3-456c-874b-6aaee269ace0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:05:33.161738 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6a4fb-17f3-456c-874b-6aaee269ace0 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:05:33.180158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 667/1349] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:05:38.189324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f4cd5-7aec-4e42-bcb4-f3bdd0338545 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:38.189324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f4cd5-7aec-4e42-bcb4-f3bdd0338545 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:38.189802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f4cd5-7aec-4e42-bcb4-f3bdd0338545 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:38.192981 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 683/1350] 10.222.0.30 () {54 vars in 921 bytes} [Thu Oct 6 08:05:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:38.199597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece48dc-2176-4ada-9589-fbf371b9f27b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:38.200344 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece48dc-2176-4ada-9589-fbf371b9f27b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:38.200791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece48dc-2176-4ada-9589-fbf371b9f27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:38.290493 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-5ece48dc-2176-4ada-9589-fbf371b9f27b None None] MFA Rules not processed for user `6074872303d3468fb566378fa34e8bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:38.292556 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5ece48dc-2176-4ada-9589-fbf371b9f27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:38.438347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 668/1351] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Oct 6 08:05:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:42.575334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=59 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:42.861629 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4b07d-1344-4a97-83a4-475d66f35bb6 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:42.862017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4b07d-1344-4a97-83a4-475d66f35bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:42.862408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4b07d-1344-4a97-83a4-475d66f35bb6 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:42.864210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 684/1352] 10.222.0.30 () {54 vars in 921 bytes} [Thu Oct 6 08:05:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:42.869429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a37d1-e143-4a3f-8622-64bcf29e843e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:42.869870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a37d1-e143-4a3f-8622-64bcf29e843e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:42.870119 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a37d1-e143-4a3f-8622-64bcf29e843e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:42.926757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:42.950092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-a32a37d1-e143-4a3f-8622-64bcf29e843e None None] MFA Rules not processed for user `6074872303d3468fb566378fa34e8bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:42.951332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a32a37d1-e143-4a3f-8622-64bcf29e843e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:43.065242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 669/1353] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Oct 6 08:05:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:44.448675 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812873c-1789-4def-b56b-89b6e5132529 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:44.449170 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812873c-1789-4def-b56b-89b6e5132529 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:44.449170 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812873c-1789-4def-b56b-89b6e5132529 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:44.450499 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 685/1354] 10.222.0.30 () {54 vars in 921 bytes} [Thu Oct 6 08:05:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:44.455816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b39644-e05d-487b-bc19-ed78a6e3901e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:44.456230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b39644-e05d-487b-bc19-ed78a6e3901e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:44.456344 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b39644-e05d-487b-bc19-ed78a6e3901e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:44.534078 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-07b39644-e05d-487b-bc19-ed78a6e3901e None None] MFA Rules not processed for user `6074872303d3468fb566378fa34e8bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:44.535330 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-07b39644-e05d-487b-bc19-ed78a6e3901e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:44.577659 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:44.642022 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 670/1355] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Oct 6 08:05:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:48.221115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eacc7f-0557-45c6-866e-5287c345e9de None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:48.222017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eacc7f-0557-45c6-866e-5287c345e9de None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:48.222391 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eacc7f-0557-45c6-866e-5287c345e9de None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:48.223950 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 686/1356] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:05:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:48.229995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eeb4ec-5ffe-4366-9d85-d3b1f9c6588e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:48.230433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eeb4ec-5ffe-4366-9d85-d3b1f9c6588e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:48.230534 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eeb4ec-5ffe-4366-9d85-d3b1f9c6588e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:48.299554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-02eeb4ec-5ffe-4366-9d85-d3b1f9c6588e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:48.300647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-02eeb4ec-5ffe-4366-9d85-d3b1f9c6588e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:48.401195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 671/1357] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:05:48.717661 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f660c5-6dae-4cae-9d12-44d397b10cc2 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:48.718065 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f660c5-6dae-4cae-9d12-44d397b10cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:48.718481 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f660c5-6dae-4cae-9d12-44d397b10cc2 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:48.719936 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 687/1358] 10.222.0.30 () {54 vars in 961 bytes} [Thu Oct 6 08:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:48.727512 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307cb55-cf4c-44f6-997b-a6a9a9fc3b6b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:48.728116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307cb55-cf4c-44f6-997b-a6a9a9fc3b6b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:48.728551 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307cb55-cf4c-44f6-997b-a6a9a9fc3b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:48.747884 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:48.814768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-4307cb55-cf4c-44f6-997b-a6a9a9fc3b6b None None] MFA Rules not processed for user `209795285384486183cda455fe316bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:48.817012 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4307cb55-cf4c-44f6-997b-a6a9a9fc3b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:48.941564 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 672/1359] 10.222.0.30 () {58 vars in 1058 bytes} [Thu Oct 6 08:05:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:05:48.950200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f660c5-6dae-4cae-9d12-44d397b10cc2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:05:48.951984 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d6f660c5-6dae-4cae-9d12-44d397b10cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:49.117474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f660c5-6dae-4cae-9d12-44d397b10cc2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:05:49.204480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:49.246057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8a29f-8e49-41ab-b851-ad7c79b8de81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:05:49.246844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8a29f-8e49-41ab-b851-ad7c79b8de81 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:49.247231 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8a29f-8e49-41ab-b851-ad7c79b8de81 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:49.247600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8a29f-8e49-41ab-b851-ad7c79b8de81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:49.249294 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dec8a29f-8e49-41ab-b851-ad7c79b8de81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:49.355028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8a29f-8e49-41ab-b851-ad7c79b8de81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:05:49.356790 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8a29f-8e49-41ab-b851-ad7c79b8de81 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:05:49.378214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 688/1360] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:05:51.785727 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d225d3-7715-4dd7-be79-878a5ce27ad9 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:51.786171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d225d3-7715-4dd7-be79-878a5ce27ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:51.786346 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d225d3-7715-4dd7-be79-878a5ce27ad9 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:51.791567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 673/1361] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:05:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:51.799237 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ec29f-da2e-4ad9-b03d-c2ca3aaa14d8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:51.799237 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ec29f-da2e-4ad9-b03d-c2ca3aaa14d8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:51.799237 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ec29f-da2e-4ad9-b03d-c2ca3aaa14d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:51.886581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8e1ec29f-da2e-4ad9-b03d-c2ca3aaa14d8 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:51.887905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8e1ec29f-da2e-4ad9-b03d-c2ca3aaa14d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:51.927520 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:51.991019 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 689/1362] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:52.286194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d225d3-7715-4dd7-be79-878a5ce27ad9 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:05:52.288008 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-42d225d3-7715-4dd7-be79-878a5ce27ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:52.357601 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:52.456062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d225d3-7715-4dd7-be79-878a5ce27ad9 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:05:52.583116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53c23e-17ad-48f3-8e1d-896b8aa09774 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:05:52.583116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53c23e-17ad-48f3-8e1d-896b8aa09774 service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:52.583896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53c23e-17ad-48f3-8e1d-896b8aa09774 service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:52.583896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53c23e-17ad-48f3-8e1d-896b8aa09774 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:52.585768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1e53c23e-17ad-48f3-8e1d-896b8aa09774 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:52.611742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:52.683882 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53c23e-17ad-48f3-8e1d-896b8aa09774 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:05:52.685536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53c23e-17ad-48f3-8e1d-896b8aa09774 service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:05:52.704417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 674/1363] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:05:55.364167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae2282-4ffc-4e8e-b8d8-ee2cf3c44e49 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:55.364702 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae2282-4ffc-4e8e-b8d8-ee2cf3c44e49 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:55.364702 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae2282-4ffc-4e8e-b8d8-ee2cf3c44e49 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:55.366983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 690/1364] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:55.372013 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f9d7b-cadd-442b-93b9-bbe1f18e8fb0 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:55.372443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f9d7b-cadd-442b-93b9-bbe1f18e8fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:55.372651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f9d7b-cadd-442b-93b9-bbe1f18e8fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:55.446218 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-be3f9d7b-cadd-442b-93b9-bbe1f18e8fb0 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:55.447340 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-be3f9d7b-cadd-442b-93b9-bbe1f18e8fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:55.567996 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:55.577122 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 675/1365] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:55.895672 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d44bff-a176-4f35-9ac2-49d0b9fdc67b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:55.896041 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d44bff-a176-4f35-9ac2-49d0b9fdc67b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:55.896218 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d44bff-a176-4f35-9ac2-49d0b9fdc67b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:55.897821 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 691/1366] 10.222.0.30 () {54 vars in 961 bytes} [Thu Oct 6 08:05:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:55.904904 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddab0c-ab03-43b5-a3fa-a7ba9e186136 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:55.905830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddab0c-ab03-43b5-a3fa-a7ba9e186136 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:55.905830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddab0c-ab03-43b5-a3fa-a7ba9e186136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:55.989993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-a3ddab0c-ab03-43b5-a3fa-a7ba9e186136 None None] MFA Rules not processed for user `209795285384486183cda455fe316bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:55.991233 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a3ddab0c-ab03-43b5-a3fa-a7ba9e186136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:56.118401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 676/1367] 10.222.0.30 () {58 vars in 1058 bytes} [Thu Oct 6 08:05:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:56.127672 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d44bff-a176-4f35-9ac2-49d0b9fdc67b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:05:56.129427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-21d44bff-a176-4f35-9ac2-49d0b9fdc67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:56.305305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d44bff-a176-4f35-9ac2-49d0b9fdc67b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:05:56.329741 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:56.437232 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92922522-2cf8-43b5-ae9e-fa7182a97ea0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:05:56.437881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92922522-2cf8-43b5-ae9e-fa7182a97ea0 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:56.438262 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92922522-2cf8-43b5-ae9e-fa7182a97ea0 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:56.438632 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92922522-2cf8-43b5-ae9e-fa7182a97ea0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:56.440339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-92922522-2cf8-43b5-ae9e-fa7182a97ea0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:56.511782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee974f-7e27-4e67-a437-2fd6138aa0f6 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:56.512318 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee974f-7e27-4e67-a437-2fd6138aa0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:56.512443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee974f-7e27-4e67-a437-2fd6138aa0f6 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:56.517772 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 677/1368] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:05:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 08:05:56.518814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248f68c-2826-4c7e-a723-d2f6e38b0f8c None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:56.519126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248f68c-2826-4c7e-a723-d2f6e38b0f8c None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:56.519610 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248f68c-2826-4c7e-a723-d2f6e38b0f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:56.541988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92922522-2cf8-43b5-ae9e-fa7182a97ea0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:05:56.543899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92922522-2cf8-43b5-ae9e-fa7182a97ea0 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:05:56.562901 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 692/1369] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:05:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:05:56.583303 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:56.593075 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-b248f68c-2826-4c7e-a723-d2f6e38b0f8c None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:56.594427 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b248f68c-2826-4c7e-a723-d2f6e38b0f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:56.703315 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 678/1370] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:57.041523 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a90145-6409-4ab4-a257-0de54161b600 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:57.041890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a90145-6409-4ab4-a257-0de54161b600 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:57.042217 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a90145-6409-4ab4-a257-0de54161b600 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:57.043858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 693/1371] 10.222.0.30 () {54 vars in 963 bytes} [Thu Oct 6 08:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:57.049585 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51db05-7e9e-4963-a587-ae40182dd53b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:57.049955 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51db05-7e9e-4963-a587-ae40182dd53b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:57.050303 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51db05-7e9e-4963-a587-ae40182dd53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:57.128652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-3c51db05-7e9e-4963-a587-ae40182dd53b None None] MFA Rules not processed for user `a899c6dabca948269f6c1370425f244b`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:57.130121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3c51db05-7e9e-4963-a587-ae40182dd53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:57.208062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:57.228584 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a90145-6409-4ab4-a257-0de54161b600 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:05:57.230269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-06a90145-6409-4ab4-a257-0de54161b600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:57.230489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 679/1372] 10.222.0.30 () {58 vars in 1060 bytes} [Thu Oct 6 08:05:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:57.261975 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:57.346532 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a90145-6409-4ab4-a257-0de54161b600 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:05:57.429106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18133a-9de2-4657-a3f7-35bc38a819f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:05:57.430331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18133a-9de2-4657-a3f7-35bc38a819f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:57.430714 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18133a-9de2-4657-a3f7-35bc38a819f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:57.431228 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18133a-9de2-4657-a3f7-35bc38a819f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:57.433105 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6f18133a-9de2-4657-a3f7-35bc38a819f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:57.526614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:57.560863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18133a-9de2-4657-a3f7-35bc38a819f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:05:57.563173 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18133a-9de2-4657-a3f7-35bc38a819f2 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:05:57.588491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 694/1373] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:05:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:05:59.053330 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162845b-c799-49a7-8893-9f3e724343c2 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:59.053884 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162845b-c799-49a7-8893-9f3e724343c2 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:59.053884 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162845b-c799-49a7-8893-9f3e724343c2 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:59.055236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 680/1374] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:05:59.061278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10706ad2-b9b8-401a-89ac-10d3f6c1cf52 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:59.061748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10706ad2-b9b8-401a-89ac-10d3f6c1cf52 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:59.061846 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10706ad2-b9b8-401a-89ac-10d3f6c1cf52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:59.127762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-10706ad2-b9b8-401a-89ac-10d3f6c1cf52 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:05:59.129214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-10706ad2-b9b8-401a-89ac-10d3f6c1cf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:59.244532 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 695/1375] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:05:59.544422 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162845b-c799-49a7-8893-9f3e724343c2 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:05:59.545961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4162845b-c799-49a7-8893-9f3e724343c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:59.705017 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162845b-c799-49a7-8893-9f3e724343c2 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:05:59.776656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:05:59.830739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92b5f2-d0b4-426a-a9b9-adad7168095a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:05:59.833703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92b5f2-d0b4-426a-a9b9-adad7168095a service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:05:59.833703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92b5f2-d0b4-426a-a9b9-adad7168095a service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:05:59.833703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92b5f2-d0b4-426a-a9b9-adad7168095a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:05:59.835743 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9b92b5f2-d0b4-426a-a9b9-adad7168095a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:05:59.948758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92b5f2-d0b4-426a-a9b9-adad7168095a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:05:59.950433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92b5f2-d0b4-426a-a9b9-adad7168095a service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:05:59.970889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 681/1376] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:05:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:06:02.540740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3410f7-94a5-4bab-ada3-ab9e70a32da6 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:02.541325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3410f7-94a5-4bab-ada3-ab9e70a32da6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:02.541325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3410f7-94a5-4bab-ada3-ab9e70a32da6 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:02.542753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 696/1377] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:02.547831 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f329f-3ad9-4176-98b0-96cc03a52c21 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:02.548237 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f329f-3ad9-4176-98b0-96cc03a52c21 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:02.548408 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f329f-3ad9-4176-98b0-96cc03a52c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:02.615592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-fe4f329f-3ad9-4176-98b0-96cc03a52c21 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:02.616772 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-fe4f329f-3ad9-4176-98b0-96cc03a52c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:02.648516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:02.720267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 682/1378] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:06:03.010651 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3410f7-94a5-4bab-ada3-ab9e70a32da6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:03.012437 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ee3410f7-94a5-4bab-ada3-ab9e70a32da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:03.055402 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:03.179657 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3410f7-94a5-4bab-ada3-ab9e70a32da6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:03.296522 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:03.302056 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed300c9d-3197-4b62-beb6-7d0d852e85ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:03.302779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed300c9d-3197-4b62-beb6-7d0d852e85ed service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:03.303165 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed300c9d-3197-4b62-beb6-7d0d852e85ed service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:03.303485 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed300c9d-3197-4b62-beb6-7d0d852e85ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:03.309157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ed300c9d-3197-4b62-beb6-7d0d852e85ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:03.405307 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed300c9d-3197-4b62-beb6-7d0d852e85ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:03.407146 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed300c9d-3197-4b62-beb6-7d0d852e85ed service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:03.429056 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 697/1379] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:06:05.968475 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8c875-e74f-4efa-a4fa-e2332e74fa9b None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:05.969133 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8c875-e74f-4efa-a4fa-e2332e74fa9b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:05.969133 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8c875-e74f-4efa-a4fa-e2332e74fa9b None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:05.970409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 683/1380] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 08:06:05.976493 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bca364-cbc1-4e5b-bd18-8a8cd4ab663d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:05.977039 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bca364-cbc1-4e5b-bd18-8a8cd4ab663d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:05.977039 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bca364-cbc1-4e5b-bd18-8a8cd4ab663d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:06.049543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f1bca364-cbc1-4e5b-bd18-8a8cd4ab663d None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:06.050824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f1bca364-cbc1-4e5b-bd18-8a8cd4ab663d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:06.130210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:06.175161 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 698/1381] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:06:06.537565 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e8c875-e74f-4efa-a4fa-e2332e74fa9b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:06.539159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a0e8c875-e74f-4efa-a4fa-e2332e74fa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:06.699404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e8c875-e74f-4efa-a4fa-e2332e74fa9b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:06.717089 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:06.829417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e53d5-a114-49f0-93f9-344b2e79eafd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:06.829926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e53d5-a114-49f0-93f9-344b2e79eafd service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:06.830203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e53d5-a114-49f0-93f9-344b2e79eafd service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:06.830469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e53d5-a114-49f0-93f9-344b2e79eafd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:06.832127 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-2b3e53d5-a114-49f0-93f9-344b2e79eafd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:06.939166 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e53d5-a114-49f0-93f9-344b2e79eafd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:06.941421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e53d5-a114-49f0-93f9-344b2e79eafd service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:06.967105 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 684/1382] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:06:09.594610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b3ac1-8c80-49e6-9f67-795316b953fa None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:09.595214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b3ac1-8c80-49e6-9f67-795316b953fa None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:09.595322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b3ac1-8c80-49e6-9f67-795316b953fa None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:09.596982 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 699/1383] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:06:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:09.603600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e1361-cc6e-4dc1-abc6-76aea3239cc8 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:09.604023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e1361-cc6e-4dc1-abc6-76aea3239cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:09.604161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e1361-cc6e-4dc1-abc6-76aea3239cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:09.629723 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:09.685934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-282e1361-cc6e-4dc1-abc6-76aea3239cc8 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:09.687560 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-282e1361-cc6e-4dc1-abc6-76aea3239cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:09.808382 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 685/1384] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:06:10.097839 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b3ac1-8c80-49e6-9f67-795316b953fa None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:10.099767 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-de4b3ac1-8c80-49e6-9f67-795316b953fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:10.273577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b3ac1-8c80-49e6-9f67-795316b953fa None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:10.346656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:10.423856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2e8bc-dfd4-48e3-b5fd-60a67ef31b04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:10.424492 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2e8bc-dfd4-48e3-b5fd-60a67ef31b04 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:10.424824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2e8bc-dfd4-48e3-b5fd-60a67ef31b04 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:10.425145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2e8bc-dfd4-48e3-b5fd-60a67ef31b04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:10.427203 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c7f2e8bc-dfd4-48e3-b5fd-60a67ef31b04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:10.528879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2e8bc-dfd4-48e3-b5fd-60a67ef31b04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:10.530491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2e8bc-dfd4-48e3-b5fd-60a67ef31b04 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:10.552150 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 700/1385] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:06:12.893901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddda26-6f51-4677-9e0c-6f4ad400980b None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:12.894410 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddda26-6f51-4677-9e0c-6f4ad400980b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:12.894410 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddda26-6f51-4677-9e0c-6f4ad400980b None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:12.895756 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 686/1386] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:12.901968 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f3a02-7163-4593-9511-a7296e2213cc None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:12.902448 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f3a02-7163-4593-9511-a7296e2213cc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:12.902573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f3a02-7163-4593-9511-a7296e2213cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:12.970258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8f3f3a02-7163-4593-9511-a7296e2213cc None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:12.971482 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8f3f3a02-7163-4593-9511-a7296e2213cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:12.981494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:13.072567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 701/1387] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:06:13.364983 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ddda26-6f51-4677-9e0c-6f4ad400980b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:13.366528 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-11ddda26-6f51-4677-9e0c-6f4ad400980b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:13.474868 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:13.521568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ddda26-6f51-4677-9e0c-6f4ad400980b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:13.525889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb2d95-7805-42c8-86c5-2aec9462e023 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:13.526203 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb2d95-7805-42c8-86c5-2aec9462e023 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:13.526474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb2d95-7805-42c8-86c5-2aec9462e023 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:13.527875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 702/1388] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:06:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 08:06:13.533279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954c597-598a-4550-9f4f-6232f1825183 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:13.533548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954c597-598a-4550-9f4f-6232f1825183 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:13.534562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954c597-598a-4550-9f4f-6232f1825183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:13.600348 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4954c597-598a-4550-9f4f-6232f1825183 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:13.601557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4954c597-598a-4550-9f4f-6232f1825183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:13.636423 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041c803-296c-4e6d-b92b-a1e32d010d89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:13.637024 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041c803-296c-4e6d-b92b-a1e32d010d89 service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:13.637653 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041c803-296c-4e6d-b92b-a1e32d010d89 service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:13.637653 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041c803-296c-4e6d-b92b-a1e32d010d89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:13.639167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-8041c803-296c-4e6d-b92b-a1e32d010d89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:13.691265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:13.701225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:13.702169 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 703/1389] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:06:13.732179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041c803-296c-4e6d-b92b-a1e32d010d89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:13.733707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041c803-296c-4e6d-b92b-a1e32d010d89 service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:13.751200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 687/1390] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:06:14.017842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ff6a4-a166-4e6e-bd37-88bf8ce6a51b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:14.018432 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ff6a4-a166-4e6e-bd37-88bf8ce6a51b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:14.018432 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ff6a4-a166-4e6e-bd37-88bf8ce6a51b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:14.019789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 704/1391] 10.222.0.30 () {54 vars in 962 bytes} [Thu Oct 6 08:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:14.026589 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1fd11-b18e-4af0-9149-efd3d511086e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:14.027138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1fd11-b18e-4af0-9149-efd3d511086e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:14.027387 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1fd11-b18e-4af0-9149-efd3d511086e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:14.111965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-8cf1fd11-b18e-4af0-9149-efd3d511086e None None] MFA Rules not processed for user `30909d152a5c4a1fac6c1a2765868177`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:14.113232 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-8cf1fd11-b18e-4af0-9149-efd3d511086e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:14.198077 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 688/1392] 10.222.0.30 () {58 vars in 1059 bytes} [Thu Oct 6 08:06:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:06:14.207832 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ff6a4-a166-4e6e-bd37-88bf8ce6a51b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:14.209429 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-591ff6a4-a166-4e6e-bd37-88bf8ce6a51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:14.318678 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ff6a4-a166-4e6e-bd37-88bf8ce6a51b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:14.393815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89fc30-2ce4-4b3d-a902-4c04b20e369e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30909d152a5c4a1fac6c1a2765868177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:14.394422 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89fc30-2ce4-4b3d-a902-4c04b20e369e service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:14.394701 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89fc30-2ce4-4b3d-a902-4c04b20e369e service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:14.394999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89fc30-2ce4-4b3d-a902-4c04b20e369e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:14.396580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fb89fc30-2ce4-4b3d-a902-4c04b20e369e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:14.441542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:14.509948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89fc30-2ce4-4b3d-a902-4c04b20e369e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:14.511464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89fc30-2ce4-4b3d-a902-4c04b20e369e service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:14.530308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 705/1393] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:06:15.513060 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41454b2a-d4cc-4d65-8b73-8193073052f1 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:15.513749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41454b2a-d4cc-4d65-8b73-8193073052f1 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:15.513749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41454b2a-d4cc-4d65-8b73-8193073052f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:15.751194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:15.751194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-41454b2a-d4cc-4d65-8b73-8193073052f1 None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:15.751194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-41454b2a-d4cc-4d65-8b73-8193073052f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:15.751194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 689/1394] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Oct 6 08:06:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:06:15.753840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434143a-e80e-4150-9d13-63f4594960c8 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:15.753840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434143a-e80e-4150-9d13-63f4594960c8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:15.753840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434143a-e80e-4150-9d13-63f4594960c8 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:15.753840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 706/1395] 10.222.0.30 () {54 vars in 943 bytes} [Thu Oct 6 08:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:15.755108 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5b7e2-8699-4211-a6a0-17e89032d8be None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:15.755108 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5b7e2-8699-4211-a6a0-17e89032d8be None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:15.755108 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5b7e2-8699-4211-a6a0-17e89032d8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:15.815675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-7ba5b7e2-8699-4211-a6a0-17e89032d8be None None] MFA Rules not processed for user `3fa1c430c9f7416eaf1d17300b827e05`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:15.816879 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7ba5b7e2-8699-4211-a6a0-17e89032d8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:15.896114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:15.928395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 690/1396] 10.222.0.30 () {58 vars in 1040 bytes} [Thu Oct 6 08:06:15 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:06:15.938285 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434143a-e80e-4150-9d13-63f4594960c8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:15.939960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0434143a-e80e-4150-9d13-63f4594960c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:16.084303 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434143a-e80e-4150-9d13-63f4594960c8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:16.096683 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0bd0b-9007-4168-9412-e56c984b40a8 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:16.096965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0bd0b-9007-4168-9412-e56c984b40a8 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:16.097236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0bd0b-9007-4168-9412-e56c984b40a8 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:16.097965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:16.098588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 691/1397] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 08:06:16.104661 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cfdf3-0d2f-4c32-9ad1-e42c30576acd None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:16.104944 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cfdf3-0d2f-4c32-9ad1-e42c30576acd None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:16.105220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cfdf3-0d2f-4c32-9ad1-e42c30576acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:16.179078 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-0c5cfdf3-0d2f-4c32-9ad1-e42c30576acd None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:16.180923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0c5cfdf3-0d2f-4c32-9ad1-e42c30576acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:16.197090 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27b0cf-9aa6-41e8-bf95-3169e5b72974 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:16.197700 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27b0cf-9aa6-41e8-bf95-3169e5b72974 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:16.197977 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27b0cf-9aa6-41e8-bf95-3169e5b72974 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:16.198378 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27b0cf-9aa6-41e8-bf95-3169e5b72974 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:16.200033 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5a27b0cf-9aa6-41e8-bf95-3169e5b72974 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:16.285167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 692/1398] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:06:16.292303 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27b0cf-9aa6-41e8-bf95-3169e5b72974 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:16.293896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27b0cf-9aa6-41e8-bf95-3169e5b72974 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:16.301921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 707/1399] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:06:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:06:16.591122 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cfdf3-0d2f-4c32-9ad1-e42c30576acd None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:16.596056 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0c5cfdf3-0d2f-4c32-9ad1-e42c30576acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:16.649650 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:16.715514 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cfdf3-0d2f-4c32-9ad1-e42c30576acd None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:16.838257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f480cc2-2873-4eb3-9097-21c88f7417e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:16.838894 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f480cc2-2873-4eb3-9097-21c88f7417e0 service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:16.839283 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f480cc2-2873-4eb3-9097-21c88f7417e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:16.839554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f480cc2-2873-4eb3-9097-21c88f7417e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:16.841182 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7f480cc2-2873-4eb3-9097-21c88f7417e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:16.895671 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:16.949169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f480cc2-2873-4eb3-9097-21c88f7417e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:16.951207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f480cc2-2873-4eb3-9097-21c88f7417e0 service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:16.971768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 693/1400] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:06:19.451788 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa736a-e740-4bc1-abd2-e4e115905d22 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:19.452310 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa736a-e740-4bc1-abd2-e4e115905d22 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:19.452310 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa736a-e740-4bc1-abd2-e4e115905d22 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:19.454054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 708/1401] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:19.459878 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb6e80-7ad1-4816-af6e-6cdf544df146 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:19.460348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb6e80-7ad1-4816-af6e-6cdf544df146 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:19.460440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb6e80-7ad1-4816-af6e-6cdf544df146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:19.540912 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-69cb6e80-7ad1-4816-af6e-6cdf544df146 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:19.542137 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-69cb6e80-7ad1-4816-af6e-6cdf544df146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:19.654842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 694/1402] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:06:19.948564 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa736a-e740-4bc1-abd2-e4e115905d22 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:19.950079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-82fa736a-e740-4bc1-abd2-e4e115905d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:20.004932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:20.122606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa736a-e740-4bc1-abd2-e4e115905d22 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:20.263739 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f0064-773f-4312-aa11-8d079966ac49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:20.264347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f0064-773f-4312-aa11-8d079966ac49 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:20.264621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f0064-773f-4312-aa11-8d079966ac49 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:20.264890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f0064-773f-4312-aa11-8d079966ac49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:20.266484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-310f0064-773f-4312-aa11-8d079966ac49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:20.276000 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:20.371739 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f0064-773f-4312-aa11-8d079966ac49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:20.373468 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f0064-773f-4312-aa11-8d079966ac49 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:20.393676 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 709/1403] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:06:22.814888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0388525-3438-4b5a-83de-2592c765509b None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:22.815454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0388525-3438-4b5a-83de-2592c765509b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:22.815570 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0388525-3438-4b5a-83de-2592c765509b None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:22.817074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 695/1404] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:22.822255 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c604aa-96d6-4e3f-a54d-7fe7cac1bb95 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:22.822678 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c604aa-96d6-4e3f-a54d-7fe7cac1bb95 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:22.822871 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c604aa-96d6-4e3f-a54d-7fe7cac1bb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:22.898561 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b7c604aa-96d6-4e3f-a54d-7fe7cac1bb95 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:22.902265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b7c604aa-96d6-4e3f-a54d-7fe7cac1bb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:22.994973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:23.022713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 710/1405] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:06:23.325255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0388525-3438-4b5a-83de-2592c765509b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:23.327178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-c0388525-3438-4b5a-83de-2592c765509b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:23.359733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:23.494424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0388525-3438-4b5a-83de-2592c765509b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:23.607700 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:23.625894 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78895aea-b580-4415-9fc9-b2303c6c1dc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:23.626506 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78895aea-b580-4415-9fc9-b2303c6c1dc4 service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:23.626780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78895aea-b580-4415-9fc9-b2303c6c1dc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:23.627081 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78895aea-b580-4415-9fc9-b2303c6c1dc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:23.628675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-78895aea-b580-4415-9fc9-b2303c6c1dc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:23.739981 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78895aea-b580-4415-9fc9-b2303c6c1dc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:23.741762 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78895aea-b580-4415-9fc9-b2303c6c1dc4 service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:23.767037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 696/1406] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:06:26.188774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45163275-ae84-4755-8dfe-3bb395536be0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:26.189356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45163275-ae84-4755-8dfe-3bb395536be0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:26.189356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45163275-ae84-4755-8dfe-3bb395536be0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:26.190884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 711/1407] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:26.196659 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a5b0e-a630-4d89-882f-efd870b83a13 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:26.197063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a5b0e-a630-4d89-882f-efd870b83a13 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:26.197289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a5b0e-a630-4d89-882f-efd870b83a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:26.270335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-663a5b0e-a630-4d89-882f-efd870b83a13 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:26.271817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-663a5b0e-a630-4d89-882f-efd870b83a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:26.341942 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:26.387900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 697/1408] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:06:26.719206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45163275-ae84-4755-8dfe-3bb395536be0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:26.721882 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-45163275-ae84-4755-8dfe-3bb395536be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:26.931359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45163275-ae84-4755-8dfe-3bb395536be0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:27.012865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:27.087967 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3624d08-4871-4af8-9069-74c9bf8bacd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:27.090630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3624d08-4871-4af8-9069-74c9bf8bacd2 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:27.090913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3624d08-4871-4af8-9069-74c9bf8bacd2 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:27.091208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3624d08-4871-4af8-9069-74c9bf8bacd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:27.092941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e3624d08-4871-4af8-9069-74c9bf8bacd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:27.222980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3624d08-4871-4af8-9069-74c9bf8bacd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:27.230130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3624d08-4871-4af8-9069-74c9bf8bacd2 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:27.263202 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 712/1409] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:06:36.382175 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=16 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:37.259897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=69 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:40.875779 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a48c8f-1bac-4cab-a07b-1aeb746418ea None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:40.876349 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a48c8f-1bac-4cab-a07b-1aeb746418ea None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:40.876349 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a48c8f-1bac-4cab-a07b-1aeb746418ea None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:40.877693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 698/1410] 10.222.0.30 () {54 vars in 918 bytes} [Thu Oct 6 08:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:40.883214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e846a-2810-4746-9f60-d91bae0aeea9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:40.883634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e846a-2810-4746-9f60-d91bae0aeea9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:40.883753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e846a-2810-4746-9f60-d91bae0aeea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:40.959225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dc4e846a-2810-4746-9f60-d91bae0aeea9 None None] MFA Rules not processed for user `6074872303d3468fb566378fa34e8bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:40.960423 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dc4e846a-2810-4746-9f60-d91bae0aeea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:41.065922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 713/1411] 10.222.0.30 () {58 vars in 1015 bytes} [Thu Oct 6 08:06:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:06:41.080694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7b8a1-db0b-45de-bb31-5b92cb31fcf8 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:41.081012 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7b8a1-db0b-45de-bb31-5b92cb31fcf8 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:41.081287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7b8a1-db0b-45de-bb31-5b92cb31fcf8 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:41.082660 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 699/1412] 10.222.0.30 () {54 vars in 918 bytes} [Thu Oct 6 08:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:41.087358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43655d1f-166f-4ddd-93c4-81d0d62fc17a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:41.087636 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43655d1f-166f-4ddd-93c4-81d0d62fc17a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:41.087905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43655d1f-166f-4ddd-93c4-81d0d62fc17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:41.163015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:41.165653 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-43655d1f-166f-4ddd-93c4-81d0d62fc17a None None] MFA Rules not processed for user `209795285384486183cda455fe316bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:41.166845 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-43655d1f-166f-4ddd-93c4-81d0d62fc17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:41.281924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 714/1413] 10.222.0.30 () {58 vars in 1015 bytes} [Thu Oct 6 08:06:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:06:43.917859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651ce48-cd03-40fe-9b82-fab8bd1085c5 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:43.918405 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651ce48-cd03-40fe-9b82-fab8bd1085c5 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:43.918405 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651ce48-cd03-40fe-9b82-fab8bd1085c5 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:43.919767 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 08:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:43.925481 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9d231-ad6c-46df-b140-91eec6ad6039 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:43.925903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9d231-ad6c-46df-b140-91eec6ad6039 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:43.925998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9d231-ad6c-46df-b140-91eec6ad6039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:43.992598 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-49b9d231-ad6c-46df-b140-91eec6ad6039 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:43.993813 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-49b9d231-ad6c-46df-b140-91eec6ad6039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:44.049565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:44.093118 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 08:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:06:44.383682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651ce48-cd03-40fe-9b82-fab8bd1085c5 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:44.385202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4651ce48-cd03-40fe-9b82-fab8bd1085c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:44.532595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651ce48-cd03-40fe-9b82-fab8bd1085c5 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:44.603910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:44.643945 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696061c7-0231-442c-8096-303a688fadfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:44.644552 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696061c7-0231-442c-8096-303a688fadfd service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:44.644829 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696061c7-0231-442c-8096-303a688fadfd service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:44.645101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696061c7-0231-442c-8096-303a688fadfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:44.646697 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-696061c7-0231-442c-8096-303a688fadfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:44.739216 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696061c7-0231-442c-8096-303a688fadfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:44.740842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696061c7-0231-442c-8096-303a688fadfd service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:44.759257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 701/1416] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:06:45.913002 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16c3e6-721f-434e-a637-77ce46fe42a4 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:45.913549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16c3e6-721f-434e-a637-77ce46fe42a4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:45.913549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16c3e6-721f-434e-a637-77ce46fe42a4 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:45.914896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 08:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:45.921927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a7f38-ee31-4d6d-9994-465804f216d1 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:45.922381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a7f38-ee31-4d6d-9994-465804f216d1 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:45.922479 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a7f38-ee31-4d6d-9994-465804f216d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:45.998339 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-9b5a7f38-ee31-4d6d-9994-465804f216d1 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:45.999578 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9b5a7f38-ee31-4d6d-9994-465804f216d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:46.025888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:46.095475 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 08:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:06:46.245721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16c3e6-721f-434e-a637-77ce46fe42a4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:46.247297 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6d16c3e6-721f-434e-a637-77ce46fe42a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:46.395242 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16c3e6-721f-434e-a637-77ce46fe42a4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:46.435606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:46.509486 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87157e4-c533-4de6-9946-f2d93e2cd727 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:46.510011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87157e4-c533-4de6-9946-f2d93e2cd727 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:46.510285 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87157e4-c533-4de6-9946-f2d93e2cd727 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:46.510552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87157e4-c533-4de6-9946-f2d93e2cd727 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:46.513322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a87157e4-c533-4de6-9946-f2d93e2cd727 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:46.602196 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87157e4-c533-4de6-9946-f2d93e2cd727 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:46.603815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87157e4-c533-4de6-9946-f2d93e2cd727 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:46.623478 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 717/1419] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:06:47.757990 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319b84b-04b7-427a-bf34-6897f890ca26 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:47.758523 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319b84b-04b7-427a-bf34-6897f890ca26 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:47.758626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319b84b-04b7-427a-bf34-6897f890ca26 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:47.759962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 08:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:47.766954 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b543f-81ff-4ea1-ab15-2fb6947e2e41 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:47.767440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b543f-81ff-4ea1-ab15-2fb6947e2e41 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:47.767555 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b543f-81ff-4ea1-ab15-2fb6947e2e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:47.821137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:47.843237 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-170b543f-81ff-4ea1-ab15-2fb6947e2e41 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:47.844371 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-170b543f-81ff-4ea1-ab15-2fb6947e2e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:47.940869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 08:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:06:48.094785 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319b84b-04b7-427a-bf34-6897f890ca26 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:48.096249 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1319b84b-04b7-427a-bf34-6897f890ca26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:48.240687 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319b84b-04b7-427a-bf34-6897f890ca26 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:48.252635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:48.353998 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c89c7-6d41-47f8-b603-405044f5f614 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:48.354518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c89c7-6d41-47f8-b603-405044f5f614 service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:48.355447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c89c7-6d41-47f8-b603-405044f5f614 service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:48.356165 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c89c7-6d41-47f8-b603-405044f5f614 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:48.357761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-fe6c89c7-6d41-47f8-b603-405044f5f614 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:48.459088 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c89c7-6d41-47f8-b603-405044f5f614 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:48.460703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c89c7-6d41-47f8-b603-405044f5f614 service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:48.479738 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 704/1422] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:06:49.594604 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb789d-6966-4c0f-94cb-cb7865cb521a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:49.595210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb789d-6966-4c0f-94cb-cb7865cb521a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:49.595210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb789d-6966-4c0f-94cb-cb7865cb521a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:49.596536 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 08:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:49.602650 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bab9f1-30a2-4e06-b94e-42e4c197dca6 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:49.603074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bab9f1-30a2-4e06-b94e-42e4c197dca6 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:49.603261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bab9f1-30a2-4e06-b94e-42e4c197dca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:49.623739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:49.680925 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-25bab9f1-30a2-4e06-b94e-42e4c197dca6 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:49.682136 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-25bab9f1-30a2-4e06-b94e-42e4c197dca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:49.778037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 08:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:06:49.929527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb789d-6966-4c0f-94cb-cb7865cb521a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:49.931071 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2bfb789d-6966-4c0f-94cb-cb7865cb521a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:50.041006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:50.077091 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb789d-6966-4c0f-94cb-cb7865cb521a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:50.193130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb5989-194b-4da4-890f-51b2c2130680 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:50.193759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb5989-194b-4da4-890f-51b2c2130680 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:50.194034 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb5989-194b-4da4-890f-51b2c2130680 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:50.194304 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb5989-194b-4da4-890f-51b2c2130680 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:50.195981 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a9eb5989-194b-4da4-890f-51b2c2130680 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:50.275901 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:50.290277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb5989-194b-4da4-890f-51b2c2130680 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:50.291895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb5989-194b-4da4-890f-51b2c2130680 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:50.310236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 720/1425] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:06:51.433523 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9510bd9-378c-4830-8794-a3163b283243 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:51.434014 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9510bd9-378c-4830-8794-a3163b283243 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:51.434014 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9510bd9-378c-4830-8794-a3163b283243 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:51.435258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 08:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:51.441466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f886f-8d74-4690-acda-3d35cd01a866 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:51.441924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f886f-8d74-4690-acda-3d35cd01a866 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:51.442022 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f886f-8d74-4690-acda-3d35cd01a866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:51.526185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ca7f886f-8d74-4690-acda-3d35cd01a866 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:51.528458 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ca7f886f-8d74-4690-acda-3d35cd01a866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:51.626852 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 08:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:06:51.779745 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9510bd9-378c-4830-8794-a3163b283243 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:51.781287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b9510bd9-378c-4830-8794-a3163b283243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:51.868441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:51.930631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9510bd9-378c-4830-8794-a3163b283243 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:52.041086 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32afb06-7baa-467d-84cc-a5ef372f071a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:52.041599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32afb06-7baa-467d-84cc-a5ef372f071a service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:52.041871 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32afb06-7baa-467d-84cc-a5ef372f071a service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:52.042163 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32afb06-7baa-467d-84cc-a5ef372f071a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:52.043803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e32afb06-7baa-467d-84cc-a5ef372f071a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:52.093256 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:52.141826 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32afb06-7baa-467d-84cc-a5ef372f071a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:52.143480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32afb06-7baa-467d-84cc-a5ef372f071a service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:52.162525 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 707/1428] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:06:53.283278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedbe66-cea6-489a-977a-08a92bf4ece4 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:53.283815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedbe66-cea6-489a-977a-08a92bf4ece4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:53.283815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedbe66-cea6-489a-977a-08a92bf4ece4 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:53.285205 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 08:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:53.291285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93e30c-91ee-4c47-8bc2-3debb5ecde30 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:53.291729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93e30c-91ee-4c47-8bc2-3debb5ecde30 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:53.291843 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93e30c-91ee-4c47-8bc2-3debb5ecde30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:53.367023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-6c93e30c-91ee-4c47-8bc2-3debb5ecde30 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:53.368243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6c93e30c-91ee-4c47-8bc2-3debb5ecde30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:53.468942 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 08:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:06:53.622174 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedbe66-cea6-489a-977a-08a92bf4ece4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:53.623747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5aedbe66-cea6-489a-977a-08a92bf4ece4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:53.672575 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:53.769392 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedbe66-cea6-489a-977a-08a92bf4ece4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:53.880544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112e67b-73da-4cd7-9db0-bb8a1a683ec4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:53.881141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112e67b-73da-4cd7-9db0-bb8a1a683ec4 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:53.881410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112e67b-73da-4cd7-9db0-bb8a1a683ec4 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:53.881670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112e67b-73da-4cd7-9db0-bb8a1a683ec4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:53.883276 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8112e67b-73da-4cd7-9db0-bb8a1a683ec4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:53.890196 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:53.970408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112e67b-73da-4cd7-9db0-bb8a1a683ec4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:53.972017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112e67b-73da-4cd7-9db0-bb8a1a683ec4 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:53.990016 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 723/1431] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:06:55.141298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c793a7-d4b6-4bec-b044-fd376dc8b580 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:55.141902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c793a7-d4b6-4bec-b044-fd376dc8b580 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:55.141902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c793a7-d4b6-4bec-b044-fd376dc8b580 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:55.143222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 08:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:55.149583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc603f9-14b0-484c-ab61-295e51eb1f3a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:55.150053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc603f9-14b0-484c-ab61-295e51eb1f3a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:55.150157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc603f9-14b0-484c-ab61-295e51eb1f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:55.225460 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2dc603f9-14b0-484c-ab61-295e51eb1f3a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:55.226643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2dc603f9-14b0-484c-ab61-295e51eb1f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:55.293586 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:55.321995 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 08:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:06:55.471476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c793a7-d4b6-4bec-b044-fd376dc8b580 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:55.472906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-50c793a7-d4b6-4bec-b044-fd376dc8b580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:55.492264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:55.622969 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c793a7-d4b6-4bec-b044-fd376dc8b580 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:55.709623 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:55.735892 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870a5d2-1bfe-43b2-9622-c4d41c606f70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:55.736514 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870a5d2-1bfe-43b2-9622-c4d41c606f70 service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:55.736790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870a5d2-1bfe-43b2-9622-c4d41c606f70 service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:55.737082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870a5d2-1bfe-43b2-9622-c4d41c606f70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:55.738752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3870a5d2-1bfe-43b2-9622-c4d41c606f70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:55.827548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870a5d2-1bfe-43b2-9622-c4d41c606f70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:55.829158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870a5d2-1bfe-43b2-9622-c4d41c606f70 service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:55.847255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 710/1434] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:06:56.975869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b281cd-d04c-49ea-ab1c-9310afe7cb42 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:56.976446 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b281cd-d04c-49ea-ab1c-9310afe7cb42 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:56.976446 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b281cd-d04c-49ea-ab1c-9310afe7cb42 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:56.977819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 08:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:06:56.984045 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cd73c-1a7c-4074-9915-9a36f1f656dc None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:56.984488 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cd73c-1a7c-4074-9915-9a36f1f656dc None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:56.984598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cd73c-1a7c-4074-9915-9a36f1f656dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:57.059573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-ba2cd73c-1a7c-4074-9915-9a36f1f656dc None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:57.061128 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-ba2cd73c-1a7c-4074-9915-9a36f1f656dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:57.102427 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:57.160572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 08:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:06:57.311096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b281cd-d04c-49ea-ab1c-9310afe7cb42 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:57.312600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-59b281cd-d04c-49ea-ab1c-9310afe7cb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:57.459362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b281cd-d04c-49ea-ab1c-9310afe7cb42 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:57.521493 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:57.581872 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3778fe-d8f1-46c0-a8fd-8d43da812d71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:57.582389 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3778fe-d8f1-46c0-a8fd-8d43da812d71 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:57.582661 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3778fe-d8f1-46c0-a8fd-8d43da812d71 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:57.582928 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3778fe-d8f1-46c0-a8fd-8d43da812d71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:57.584532 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6e3778fe-d8f1-46c0-a8fd-8d43da812d71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:57.673901 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3778fe-d8f1-46c0-a8fd-8d43da812d71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:57.675524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3778fe-d8f1-46c0-a8fd-8d43da812d71 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:57.693774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 726/1437] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:06:58.873055 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f959a1f-b87f-457f-b0ba-72bda8067510 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:58.873615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f959a1f-b87f-457f-b0ba-72bda8067510 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:58.873615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f959a1f-b87f-457f-b0ba-72bda8067510 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:58.874980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 08:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 08:06:58.881353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d4f21-c9fa-45fa-bfd4-3bfcd7441501 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:58.881790 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d4f21-c9fa-45fa-bfd4-3bfcd7441501 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:58.881906 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d4f21-c9fa-45fa-bfd4-3bfcd7441501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:58.957042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-133d4f21-c9fa-45fa-bfd4-3bfcd7441501 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:06:58.958154 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-133d4f21-c9fa-45fa-bfd4-3bfcd7441501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:58.960165 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:59.055745 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 08:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:06:59.208187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f959a1f-b87f-457f-b0ba-72bda8067510 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:06:59.209708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6f959a1f-b87f-457f-b0ba-72bda8067510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:59.353905 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f959a1f-b87f-457f-b0ba-72bda8067510 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:06:59.381655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:06:59.466888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae01c4-c926-46dd-a040-fdad7e366591 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:06:59.467460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae01c4-c926-46dd-a040-fdad7e366591 service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:06:59.467736 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae01c4-c926-46dd-a040-fdad7e366591 service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:06:59.468006 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae01c4-c926-46dd-a040-fdad7e366591 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:06:59.469602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-91ae01c4-c926-46dd-a040-fdad7e366591 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:06:59.560875 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae01c4-c926-46dd-a040-fdad7e366591 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:06:59.562506 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae01c4-c926-46dd-a040-fdad7e366591 service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:06:59.580694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 713/1440] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:07:01.679054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ebb90-5e73-41e7-ad3c-06cd72c651ec None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:01.679625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ebb90-5e73-41e7-ad3c-06cd72c651ec None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:01.679625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ebb90-5e73-41e7-ad3c-06cd72c651ec None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:01.680960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 08:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:07:01.687102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff10f7-7eef-49b2-8a0d-e1eceed1805e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:01.687517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff10f7-7eef-49b2-8a0d-e1eceed1805e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:01.687623 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff10f7-7eef-49b2-8a0d-e1eceed1805e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:01.727758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:01.752706 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-58ff10f7-7eef-49b2-8a0d-e1eceed1805e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:07:01.753917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-58ff10f7-7eef-49b2-8a0d-e1eceed1805e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:01.850224 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 08:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:07:02.134132 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ebb90-5e73-41e7-ad3c-06cd72c651ec None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:07:02.135713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9c2ebb90-5e73-41e7-ad3c-06cd72c651ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:02.241115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ebb90-5e73-41e7-ad3c-06cd72c651ec None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:07:02.273465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:02.313829 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff240a-4ae0-4ec2-855b-f0b246c16e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:07:02.314428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff240a-4ae0-4ec2-855b-f0b246c16e2b service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:02.314703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff240a-4ae0-4ec2-855b-f0b246c16e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:02.315003 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff240a-4ae0-4ec2-855b-f0b246c16e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:02.316588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e5ff240a-4ae0-4ec2-855b-f0b246c16e2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:02.430752 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff240a-4ae0-4ec2-855b-f0b246c16e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:07:02.432136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff240a-4ae0-4ec2-855b-f0b246c16e2b service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:07:02.451503 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 729/1443] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:07:04.905781 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c46a6d-43b8-4be0-9464-898bc9a071b9 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:04.906326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c46a6d-43b8-4be0-9464-898bc9a071b9 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:04.906326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c46a6d-43b8-4be0-9464-898bc9a071b9 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:04.907690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 08:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:07:04.913122 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f94e60-7ce4-4c1e-8a4a-6c4752cba0c7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:04.913541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f94e60-7ce4-4c1e-8a4a-6c4752cba0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:04.913673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f94e60-7ce4-4c1e-8a4a-6c4752cba0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:04.978876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-23f94e60-7ce4-4c1e-8a4a-6c4752cba0c7 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:07:04.980698 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-23f94e60-7ce4-4c1e-8a4a-6c4752cba0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:04.993518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:05.077012 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 08:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:07:05.367921 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c46a6d-43b8-4be0-9464-898bc9a071b9 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:07:05.369460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-86c46a6d-43b8-4be0-9464-898bc9a071b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:05.472423 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c46a6d-43b8-4be0-9464-898bc9a071b9 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:07:05.485309 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:05.555699 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc59e49-d71a-4664-8699-3d63d0ffd4c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:07:05.556312 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc59e49-d71a-4664-8699-3d63d0ffd4c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:05.556661 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc59e49-d71a-4664-8699-3d63d0ffd4c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:05.557170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc59e49-d71a-4664-8699-3d63d0ffd4c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:05.559857 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-afc59e49-d71a-4664-8699-3d63d0ffd4c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:05.672196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc59e49-d71a-4664-8699-3d63d0ffd4c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:07:05.673552 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc59e49-d71a-4664-8699-3d63d0ffd4c9 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:07:05.692169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 716/1446] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:07:08.362356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e61e4-09b0-4f73-b8a8-dfd48efde295 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:08.362970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e61e4-09b0-4f73-b8a8-dfd48efde295 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:08.362970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e61e4-09b0-4f73-b8a8-dfd48efde295 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:08.364308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 08:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:07:08.369716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94addee7-8833-42b0-884a-1f34734ec4ae None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:08.370147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94addee7-8833-42b0-884a-1f34734ec4ae None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:08.370249 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94addee7-8833-42b0-884a-1f34734ec4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:08.419572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:08.437121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-94addee7-8833-42b0-884a-1f34734ec4ae None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:07:08.438320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-94addee7-8833-42b0-884a-1f34734ec4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:08.535063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 08:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:07:08.821095 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e61e4-09b0-4f73-b8a8-dfd48efde295 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:07:08.822667 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1f1e61e4-09b0-4f73-b8a8-dfd48efde295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:08.928296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e61e4-09b0-4f73-b8a8-dfd48efde295 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:07:08.968966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:09.001634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3fc1b-e701-4519-aa3f-a3f36387596a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:07:09.002272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3fc1b-e701-4519-aa3f-a3f36387596a service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:09.002769 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3fc1b-e701-4519-aa3f-a3f36387596a service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:09.003289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3fc1b-e701-4519-aa3f-a3f36387596a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:09.005503 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-faf3fc1b-e701-4519-aa3f-a3f36387596a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:09.114344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3fc1b-e701-4519-aa3f-a3f36387596a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:07:09.115727 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3fc1b-e701-4519-aa3f-a3f36387596a service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:07:09.134167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 732/1449] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:07:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:07:11.405878 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850c235-f170-4483-a1aa-8cfbbd52f800 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:11.406329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850c235-f170-4483-a1aa-8cfbbd52f800 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:11.406424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850c235-f170-4483-a1aa-8cfbbd52f800 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:11.407735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 08:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:07:11.412952 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597a952-7f56-422e-b6db-1edd02b06bb5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:11.413375 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597a952-7f56-422e-b6db-1edd02b06bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:11.413491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597a952-7f56-422e-b6db-1edd02b06bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:11.480995 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7597a952-7f56-422e-b6db-1edd02b06bb5 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:07:11.482260 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7597a952-7f56-422e-b6db-1edd02b06bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:11.504000 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:11.581158 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 08:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:07:11.870053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850c235-f170-4483-a1aa-8cfbbd52f800 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:07:11.871929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4850c235-f170-4483-a1aa-8cfbbd52f800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:11.973160 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850c235-f170-4483-a1aa-8cfbbd52f800 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:07:11.992800 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:12.044510 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc137ce-bae1-46ad-8779-14117089d066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:07:12.059803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc137ce-bae1-46ad-8779-14117089d066 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:12.059803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc137ce-bae1-46ad-8779-14117089d066 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:12.059803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc137ce-bae1-46ad-8779-14117089d066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:12.059803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-ffc137ce-bae1-46ad-8779-14117089d066 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:12.155930 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc137ce-bae1-46ad-8779-14117089d066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:07:12.157370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc137ce-bae1-46ad-8779-14117089d066 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:07:12.174953 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 719/1452] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:07:14.383950 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3db88-8633-402c-93bc-168d33bbc872 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:14.384464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3db88-8633-402c-93bc-168d33bbc872 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:14.384464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3db88-8633-402c-93bc-168d33bbc872 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:14.386337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 08:07:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:07:14.391549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71521582-8758-42c5-9d67-f7683cde5a8e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:14.391983 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71521582-8758-42c5-9d67-f7683cde5a8e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:14.392082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71521582-8758-42c5-9d67-f7683cde5a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:14.455281 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:14.457869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-71521582-8758-42c5-9d67-f7683cde5a8e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:07:14.459095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-71521582-8758-42c5-9d67-f7683cde5a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:14.554372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 08:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:07:14.839322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3db88-8633-402c-93bc-168d33bbc872 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:07:14.840907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ace3db88-8633-402c-93bc-168d33bbc872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:14.946705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3db88-8633-402c-93bc-168d33bbc872 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:07:14.996929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:15.020708 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664517da-55a3-4b95-89af-6a58cc05b737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:07:15.021311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664517da-55a3-4b95-89af-6a58cc05b737 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:15.021586 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664517da-55a3-4b95-89af-6a58cc05b737 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:15.021859 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664517da-55a3-4b95-89af-6a58cc05b737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:15.023475 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-664517da-55a3-4b95-89af-6a58cc05b737 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:15.131911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664517da-55a3-4b95-89af-6a58cc05b737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:07:15.133272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664517da-55a3-4b95-89af-6a58cc05b737 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:07:15.153911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 735/1455] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:07:17.472152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10449ee4-98da-4f27-8e68-e08a5fb0b974 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:17.472152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10449ee4-98da-4f27-8e68-e08a5fb0b974 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:17.472152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10449ee4-98da-4f27-8e68-e08a5fb0b974 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:17.472152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 08:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:07:17.476374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a4701-d6a8-47bf-af05-3e4243c612fe None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:17.476758 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a4701-d6a8-47bf-af05-3e4243c612fe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:17.476906 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a4701-d6a8-47bf-af05-3e4243c612fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:17.545005 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-901a4701-d6a8-47bf-af05-3e4243c612fe None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:07:17.546210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-901a4701-d6a8-47bf-af05-3e4243c612fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:17.576830 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:17.647740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 08:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:07:17.936876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10449ee4-98da-4f27-8e68-e08a5fb0b974 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:07:17.938409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-10449ee4-98da-4f27-8e68-e08a5fb0b974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:18.041697 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10449ee4-98da-4f27-8e68-e08a5fb0b974 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:07:18.071612 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:18.116809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e098c15-81ca-401f-a045-229270d5c652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:07:18.117404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e098c15-81ca-401f-a045-229270d5c652 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:18.117675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e098c15-81ca-401f-a045-229270d5c652 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:18.117934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e098c15-81ca-401f-a045-229270d5c652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:18.119527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5e098c15-81ca-401f-a045-229270d5c652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:18.228998 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e098c15-81ca-401f-a045-229270d5c652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:07:18.230923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e098c15-81ca-401f-a045-229270d5c652 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:07:18.249217 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 722/1458] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:07:20.466580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4a537-1e86-4dd9-a83c-99248d51c5c9 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:20.467146 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4a537-1e86-4dd9-a83c-99248d51c5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:20.467146 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4a537-1e86-4dd9-a83c-99248d51c5c9 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:20.468419 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 08:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:07:20.473666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfbef8-f11c-433a-b32a-13a86da096aa None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:20.478865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfbef8-f11c-433a-b32a-13a86da096aa None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:20.478865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfbef8-f11c-433a-b32a-13a86da096aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:20.540619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-0ddfbef8-f11c-433a-b32a-13a86da096aa None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:07:20.541991 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0ddfbef8-f11c-433a-b32a-13a86da096aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:20.550290 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:20.638783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 08:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:07:20.920523 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4a537-1e86-4dd9-a83c-99248d51c5c9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:07:20.922035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b8e4a537-1e86-4dd9-a83c-99248d51c5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:21.031223 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4a537-1e86-4dd9-a83c-99248d51c5c9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:07:21.084936 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:21.102489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa80600-3bd6-45a3-a8ee-24bb9984f6aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:07:21.106007 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa80600-3bd6-45a3-a8ee-24bb9984f6aa service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:21.106007 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa80600-3bd6-45a3-a8ee-24bb9984f6aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:21.106007 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa80600-3bd6-45a3-a8ee-24bb9984f6aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:21.106007 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-efa80600-3bd6-45a3-a8ee-24bb9984f6aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:21.213418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa80600-3bd6-45a3-a8ee-24bb9984f6aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:07:21.214784 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa80600-3bd6-45a3-a8ee-24bb9984f6aa service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:07:21.232389 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 738/1461] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:07:23.464744 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3872e-d64f-4376-98d6-059a86e037a3 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:23.465246 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3872e-d64f-4376-98d6-059a86e037a3 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:23.465246 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3872e-d64f-4376-98d6-059a86e037a3 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:23.466564 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 08:07:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 08:07:23.471920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e25259-3494-4e1f-8a92-5b62a741854a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:23.473748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e25259-3494-4e1f-8a92-5b62a741854a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:23.473748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e25259-3494-4e1f-8a92-5b62a741854a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:23.540031 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b6e25259-3494-4e1f-8a92-5b62a741854a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:07:23.541229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b6e25259-3494-4e1f-8a92-5b62a741854a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:23.590365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:23.649527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 08:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:07:23.935999 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3872e-d64f-4376-98d6-059a86e037a3 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:07:23.937521 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1cc3872e-d64f-4376-98d6-059a86e037a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:24.041140 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3872e-d64f-4376-98d6-059a86e037a3 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:07:24.079707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:24.115563 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f33c08-b497-480e-8ed4-7dc772dfb8c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:07:24.116172 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f33c08-b497-480e-8ed4-7dc772dfb8c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:24.116452 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f33c08-b497-480e-8ed4-7dc772dfb8c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:24.116719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f33c08-b497-480e-8ed4-7dc772dfb8c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:24.118318 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a6f33c08-b497-480e-8ed4-7dc772dfb8c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:24.231414 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f33c08-b497-480e-8ed4-7dc772dfb8c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:07:24.232740 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f33c08-b497-480e-8ed4-7dc772dfb8c1 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:07:24.250373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 725/1464] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:07:26.415037 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3cb04-625b-4bc6-8e17-d40f80af988d None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:26.415573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3cb04-625b-4bc6-8e17-d40f80af988d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:26.415573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3cb04-625b-4bc6-8e17-d40f80af988d None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:26.417463 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 08:07:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:07:26.423123 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34f72c-b01e-4212-8632-e2e54bb0f30e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:26.423527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34f72c-b01e-4212-8632-e2e54bb0f30e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:26.423692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34f72c-b01e-4212-8632-e2e54bb0f30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:26.489955 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-ce34f72c-b01e-4212-8632-e2e54bb0f30e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:07:26.491243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-ce34f72c-b01e-4212-8632-e2e54bb0f30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:26.509498 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:26.587441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 08:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:07:26.873400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3cb04-625b-4bc6-8e17-d40f80af988d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:07:26.875026 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-24a3cb04-625b-4bc6-8e17-d40f80af988d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:26.978034 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3cb04-625b-4bc6-8e17-d40f80af988d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:07:27.045863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:27.052228 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a123a7-0f3a-4903-809d-9cd3a12689db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:07:27.052827 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a123a7-0f3a-4903-809d-9cd3a12689db service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:27.053099 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a123a7-0f3a-4903-809d-9cd3a12689db service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:27.053363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a123a7-0f3a-4903-809d-9cd3a12689db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:27.054955 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-55a123a7-0f3a-4903-809d-9cd3a12689db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:27.164337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a123a7-0f3a-4903-809d-9cd3a12689db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:07:27.165771 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a123a7-0f3a-4903-809d-9cd3a12689db service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:07:27.185183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 741/1467] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:07:29.459281 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d8515-350a-4bbc-8039-86d6c0bba1b8 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:29.459822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d8515-350a-4bbc-8039-86d6c0bba1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:29.459822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d8515-350a-4bbc-8039-86d6c0bba1b8 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:29.461129 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 08:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:07:29.467188 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fc103-2cf6-4f16-b5b7-dac9906c51bb None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:29.478217 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fc103-2cf6-4f16-b5b7-dac9906c51bb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:29.478217 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fc103-2cf6-4f16-b5b7-dac9906c51bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:29.533615 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3b9fc103-2cf6-4f16-b5b7-dac9906c51bb None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:07:29.534833 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3b9fc103-2cf6-4f16-b5b7-dac9906c51bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:29.582703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:29.634557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 08:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:07:29.917586 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d8515-350a-4bbc-8039-86d6c0bba1b8 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:07:29.919245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-642d8515-350a-4bbc-8039-86d6c0bba1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:30.020815 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d8515-350a-4bbc-8039-86d6c0bba1b8 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:07:30.068625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:30.094911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6b12f-12bb-4fc7-af7f-e86d72aecebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:07:30.095544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6b12f-12bb-4fc7-af7f-e86d72aecebd service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:30.095824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6b12f-12bb-4fc7-af7f-e86d72aecebd service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:30.096104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6b12f-12bb-4fc7-af7f-e86d72aecebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:30.097702 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e7c6b12f-12bb-4fc7-af7f-e86d72aecebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:30.210070 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6b12f-12bb-4fc7-af7f-e86d72aecebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:07:30.211647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6b12f-12bb-4fc7-af7f-e86d72aecebd service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:07:30.229915 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 728/1470] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:07:39.632981 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=22 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:40.227310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=66 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:50.412777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e096a-e103-4c37-a0e6-7572eb260c3b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:50.413339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e096a-e103-4c37-a0e6-7572eb260c3b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:50.413339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e096a-e103-4c37-a0e6-7572eb260c3b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:50.414577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 743/1471] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:07:50.420633 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67fd1d-5013-4e19-b6cc-6bfa59bc8ffe None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:50.421092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67fd1d-5013-4e19-b6cc-6bfa59bc8ffe None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:50.421195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67fd1d-5013-4e19-b6cc-6bfa59bc8ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:50.613615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-7f67fd1d-5013-4e19-b6cc-6bfa59bc8ffe None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:07:50.614816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7f67fd1d-5013-4e19-b6cc-6bfa59bc8ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:50.710742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 729/1472] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:07:51.010369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a75a17-797b-4a0b-85be-34283c57c8ee None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:51.010650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a75a17-797b-4a0b-85be-34283c57c8ee None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:51.010919 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a75a17-797b-4a0b-85be-34283c57c8ee None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:51.012351 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 744/1473] 10.222.0.30 () {54 vars in 962 bytes} [Thu Oct 6 08:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:07:51.017499 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305aa7c-5b45-4ae9-b1b9-1e1d9bd628ac None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:51.017785 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305aa7c-5b45-4ae9-b1b9-1e1d9bd628ac None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:51.018065 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305aa7c-5b45-4ae9-b1b9-1e1d9bd628ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:51.093329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-3305aa7c-5b45-4ae9-b1b9-1e1d9bd628ac None None] MFA Rules not processed for user `30909d152a5c4a1fac6c1a2765868177`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:07:51.094522 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3305aa7c-5b45-4ae9-b1b9-1e1d9bd628ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:51.114734 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:51.170886 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 730/1474] 10.222.0.30 () {58 vars in 1059 bytes} [Thu Oct 6 08:07:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:07:51.178687 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a75a17-797b-4a0b-85be-34283c57c8ee None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:07:51.180227 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b0a75a17-797b-4a0b-85be-34283c57c8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:51.284472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a75a17-797b-4a0b-85be-34283c57c8ee None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:07:51.358462 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6feef0-9b66-4705-a36a-745ab2c1c048 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30909d152a5c4a1fac6c1a2765868177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:07:51.359205 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6feef0-9b66-4705-a36a-745ab2c1c048 service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:51.359483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6feef0-9b66-4705-a36a-745ab2c1c048 service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:51.359747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6feef0-9b66-4705-a36a-745ab2c1c048 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:51.361317 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1f6feef0-9b66-4705-a36a-745ab2c1c048 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:51.414279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:51.469357 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6feef0-9b66-4705-a36a-745ab2c1c048 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:07:51.470825 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6feef0-9b66-4705-a36a-745ab2c1c048 service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:07:51.490236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 745/1475] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:07:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:07:53.810113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471d4b8-720b-4a41-a92b-63209320838d None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:53.810682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471d4b8-720b-4a41-a92b-63209320838d None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:53.810682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471d4b8-720b-4a41-a92b-63209320838d None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:53.812041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 731/1476] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:07:53.817393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea6362-f804-4cdd-81db-96ff8b8c56d5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:53.817826 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea6362-f804-4cdd-81db-96ff8b8c56d5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:53.817934 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea6362-f804-4cdd-81db-96ff8b8c56d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:53.887380 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e4ea6362-f804-4cdd-81db-96ff8b8c56d5 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:07:53.888638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e4ea6362-f804-4cdd-81db-96ff8b8c56d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:53.991298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 746/1477] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:07:54.272124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471d4b8-720b-4a41-a92b-63209320838d None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:07:54.273642 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3471d4b8-720b-4a41-a92b-63209320838d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:54.396800 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471d4b8-720b-4a41-a92b-63209320838d None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:07:54.445626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:54.490846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee093e-5e54-452e-942a-daa51f873dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:07:54.491476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee093e-5e54-452e-942a-daa51f873dfd admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:54.491751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee093e-5e54-452e-942a-daa51f873dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:54.492016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee093e-5e54-452e-942a-daa51f873dfd admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:54.492643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee093e-5e54-452e-942a-daa51f873dfd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:07:54.494625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: WARNING py.warnings [None req-86ee093e-5e54-452e-942a-daa51f873dfd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 08:07:54.494625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: warnings.warn(msg) Oct 06 08:07:54.494625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: Oct 06 08:07:54.495878 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee093e-5e54-452e-942a-daa51f873dfd admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:07:54.515823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 732/1478] 10.222.0.30 () {56 vars in 1108 bytes} [Thu Oct 6 08:07:54 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:07:56.698755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592e63a-0740-489a-82e6-520953040462 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:56.699360 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592e63a-0740-489a-82e6-520953040462 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:56.699360 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592e63a-0740-489a-82e6-520953040462 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:56.700675 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 747/1479] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:07:56.705994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14e3b5-397a-4d2c-a862-03258718cc69 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:56.706484 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14e3b5-397a-4d2c-a862-03258718cc69 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:56.706588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14e3b5-397a-4d2c-a862-03258718cc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:56.772345 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-3a14e3b5-397a-4d2c-a862-03258718cc69 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:07:56.773528 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3a14e3b5-397a-4d2c-a862-03258718cc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:56.866212 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 733/1480] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:07:57.159403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592e63a-0740-489a-82e6-520953040462 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:07:57.160910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8592e63a-0740-489a-82e6-520953040462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:57.170711 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:57.305145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592e63a-0740-489a-82e6-520953040462 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:07:57.385113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:07:57.417425 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5992e5e-6f54-4ef8-8507-5e1610e600c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:07:57.418021 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5992e5e-6f54-4ef8-8507-5e1610e600c5 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:57.418324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5992e5e-6f54-4ef8-8507-5e1610e600c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:57.418598 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5992e5e-6f54-4ef8-8507-5e1610e600c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:57.420877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e5992e5e-6f54-4ef8-8507-5e1610e600c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:57.510777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5992e5e-6f54-4ef8-8507-5e1610e600c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:07:57.512472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5992e5e-6f54-4ef8-8507-5e1610e600c5 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:07:57.530164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 748/1481] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:07:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:07:59.867281 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac7c39-47ca-4326-8ad4-d9a3ca0caf9e None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:59.867847 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac7c39-47ca-4326-8ad4-d9a3ca0caf9e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:59.867847 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac7c39-47ca-4326-8ad4-d9a3ca0caf9e None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:59.869365 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 734/1482] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:07:59.874634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c0d9c-3dc6-4cdc-ba81-1a3b74b3c908 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:07:59.875068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c0d9c-3dc6-4cdc-ba81-1a3b74b3c908 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:07:59.875203 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c0d9c-3dc6-4cdc-ba81-1a3b74b3c908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:07:59.943196 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-026c0d9c-3dc6-4cdc-ba81-1a3b74b3c908 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:07:59.944396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-026c0d9c-3dc6-4cdc-ba81-1a3b74b3c908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:07:59.991147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:08:00.040866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 749/1483] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:08:00.331875 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac7c39-47ca-4326-8ad4-d9a3ca0caf9e None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:08:00.333475 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-dbac7c39-47ca-4326-8ad4-d9a3ca0caf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:08:00.357489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:08:00.435423 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac7c39-47ca-4326-8ad4-d9a3ca0caf9e None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:08:00.508041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2522b-3b19-4ad2-aa36-8ad3e0f8317b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:08:00.509557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2522b-3b19-4ad2-aa36-8ad3e0f8317b service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:08:00.509557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2522b-3b19-4ad2-aa36-8ad3e0f8317b service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:08:00.509821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2522b-3b19-4ad2-aa36-8ad3e0f8317b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:08:00.511481 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-24f2522b-3b19-4ad2-aa36-8ad3e0f8317b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:08:00.590771 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:08:00.624488 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2522b-3b19-4ad2-aa36-8ad3e0f8317b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:08:00.625939 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2522b-3b19-4ad2-aa36-8ad3e0f8317b service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:08:00.644681 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 735/1484] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:08:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:08:02.995268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4802e-d51c-48f3-a73b-fe82a94312ee None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:08:02.995816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4802e-d51c-48f3-a73b-fe82a94312ee None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:08:02.995816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4802e-d51c-48f3-a73b-fe82a94312ee None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:08:02.997157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 750/1485] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:08:03.002351 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee86ce-f68e-4d47-9eb8-814a021091b9 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:08:03.002786 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee86ce-f68e-4d47-9eb8-814a021091b9 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:08:03.002884 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee86ce-f68e-4d47-9eb8-814a021091b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:08:03.069687 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-9fee86ce-f68e-4d47-9eb8-814a021091b9 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:08:03.070931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9fee86ce-f68e-4d47-9eb8-814a021091b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:08:03.182658 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 736/1486] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:08:03.472888 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4802e-d51c-48f3-a73b-fe82a94312ee None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:08:03.474449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a6d4802e-d51c-48f3-a73b-fe82a94312ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:08:03.581540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4802e-d51c-48f3-a73b-fe82a94312ee None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:08:03.654567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:08:03.656231 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0ffbc-4776-466a-bc8b-165f141eec61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:08:03.656803 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0ffbc-4776-466a-bc8b-165f141eec61 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:08:03.657078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0ffbc-4776-466a-bc8b-165f141eec61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:08:03.657373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0ffbc-4776-466a-bc8b-165f141eec61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:08:03.659005 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c0c0ffbc-4776-466a-bc8b-165f141eec61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:08:03.772626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0ffbc-4776-466a-bc8b-165f141eec61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:08:03.774725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0ffbc-4776-466a-bc8b-165f141eec61 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:08:03.793240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 751/1487] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:08:06.529583 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2fff8-89ca-441a-bb90-8315f38e81d0 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:08:06.530121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2fff8-89ca-441a-bb90-8315f38e81d0 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:08:06.530121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2fff8-89ca-441a-bb90-8315f38e81d0 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:08:06.531516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 737/1488] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:08:06.536860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e0329-c030-43e5-832a-e1aa07b1f120 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:08:06.537319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e0329-c030-43e5-832a-e1aa07b1f120 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:08:06.537424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e0329-c030-43e5-832a-e1aa07b1f120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:08:06.604827 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fa3e0329-c030-43e5-832a-e1aa07b1f120 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:08:06.606082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fa3e0329-c030-43e5-832a-e1aa07b1f120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:08:06.660806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:08:06.705396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 752/1489] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:08:07.008120 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2fff8-89ca-441a-bb90-8315f38e81d0 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:08:07.010253 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7fc2fff8-89ca-441a-bb90-8315f38e81d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:08:07.045554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:08:07.114092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2fff8-89ca-441a-bb90-8315f38e81d0 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:08:07.187590 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb50dc9-fa37-404f-ac3f-6a9f8307b447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:08:07.188199 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb50dc9-fa37-404f-ac3f-6a9f8307b447 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:08:07.188455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb50dc9-fa37-404f-ac3f-6a9f8307b447 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:08:07.188718 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb50dc9-fa37-404f-ac3f-6a9f8307b447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:08:07.190290 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0eb50dc9-fa37-404f-ac3f-6a9f8307b447 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:08:07.279931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:08:07.308668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb50dc9-fa37-404f-ac3f-6a9f8307b447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:08:07.310110 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb50dc9-fa37-404f-ac3f-6a9f8307b447 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:08:07.329525 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 738/1490] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:08:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:08:09.973051 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246fde1-20a2-4fd0-bc82-e9bc00f40fc0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:08:09.973601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246fde1-20a2-4fd0-bc82-e9bc00f40fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:08:09.973601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246fde1-20a2-4fd0-bc82-e9bc00f40fc0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:08:09.975016 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 753/1491] 10.222.0.30 () {54 vars in 927 bytes} [Thu Oct 6 08:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:08:09.980295 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cdf6e-1c59-4ec0-8753-d0d5883c74e3 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:08:09.980722 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cdf6e-1c59-4ec0-8753-d0d5883c74e3 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:08:09.980833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cdf6e-1c59-4ec0-8753-d0d5883c74e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:08:10.045368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-120cdf6e-1c59-4ec0-8753-d0d5883c74e3 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:08:10.046565 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-120cdf6e-1c59-4ec0-8753-d0d5883c74e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:08:10.142704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 739/1492] 10.222.0.30 () {58 vars in 1024 bytes} [Thu Oct 6 08:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:08:10.431632 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246fde1-20a2-4fd0-bc82-e9bc00f40fc0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:08:10.433165 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c246fde1-20a2-4fd0-bc82-e9bc00f40fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:08:10.536966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246fde1-20a2-4fd0-bc82-e9bc00f40fc0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:08:10.610323 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a9dad-d33c-4f5f-93b3-278de2249eda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:08:10.610916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a9dad-d33c-4f5f-93b3-278de2249eda service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:08:10.611222 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a9dad-d33c-4f5f-93b3-278de2249eda service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:08:10.611492 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a9dad-d33c-4f5f-93b3-278de2249eda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:08:10.613074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-408a9dad-d33c-4f5f-93b3-278de2249eda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:08:10.621849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:08:10.726164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a9dad-d33c-4f5f-93b3-278de2249eda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:08:10.727755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a9dad-d33c-4f5f-93b3-278de2249eda service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:08:10.746988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 754/1493] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:08:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:08:20.141166 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=82 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:08:20.744858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=50 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:09:57.255748 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ccb5a-10ff-4a99-a9cf-e61e37e4e2cf None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:57.256336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ccb5a-10ff-4a99-a9cf-e61e37e4e2cf None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:57.256336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ccb5a-10ff-4a99-a9cf-e61e37e4e2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:57.335752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-277ccb5a-10ff-4a99-a9cf-e61e37e4e2cf None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:09:57.336940 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-277ccb5a-10ff-4a99-a9cf-e61e37e4e2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:09:57.436096 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 740/1494] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:09:57.444740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e6bc9-54e9-4015-9a9c-9d394f49fd81 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:57.445220 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e6bc9-54e9-4015-9a9c-9d394f49fd81 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:57.445323 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e6bc9-54e9-4015-9a9c-9d394f49fd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:57.465868 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a27e6bc9-54e9-4015-9a9c-9d394f49fd81 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:09:57.467105 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a27e6bc9-54e9-4015-9a9c-9d394f49fd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:09:57.475846 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 755/1495] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:09:57.480394 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ccb5a-10ff-4a99-a9cf-e61e37e4e2cf None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:09:57.489915 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ccb5a-10ff-4a99-a9cf-e61e37e4e2cf None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:09:57.494461 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a294c1-14f5-4857-94f6-9b82cdeca4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:09:57.495085 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a294c1-14f5-4857-94f6-9b82cdeca4c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:57.495348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a294c1-14f5-4857-94f6-9b82cdeca4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:57.495611 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a294c1-14f5-4857-94f6-9b82cdeca4c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:57.496242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a294c1-14f5-4857-94f6-9b82cdeca4c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:09:57.496533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a294c1-14f5-4857-94f6-9b82cdeca4c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:09:57.497536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: WARNING py.warnings [None req-51a294c1-14f5-4857-94f6-9b82cdeca4c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 06 08:09:57.497536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: warnings.warn(msg) Oct 06 08:09:57.497536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: Oct 06 08:09:57.498412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a294c1-14f5-4857-94f6-9b82cdeca4c0 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:09:57.510761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 741/1496] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:09:57.519756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e6bc9-54e9-4015-9a9c-9d394f49fd81 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:09:57.529187 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e6bc9-54e9-4015-9a9c-9d394f49fd81 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:09:57.533896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59b329-2266-45f1-90d0-eaa7632bf3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:09:57.534515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59b329-2266-45f1-90d0-eaa7632bf3da admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:57.534792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59b329-2266-45f1-90d0-eaa7632bf3da admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:57.535112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59b329-2266-45f1-90d0-eaa7632bf3da admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:57.536785 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec59b329-2266-45f1-90d0-eaa7632bf3da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:09:57.539040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-ec59b329-2266-45f1-90d0-eaa7632bf3da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 08:09:57.539040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Oct 06 08:09:57.539040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: Oct 06 08:09:57.541863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec59b329-2266-45f1-90d0-eaa7632bf3da admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:09:57.658631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 756/1497] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:09:57 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:09:57.663022 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a294c1-14f5-4857-94f6-9b82cdeca4c0 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:09:57.673406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a294c1-14f5-4857-94f6-9b82cdeca4c0 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:09:57.723195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:09:57.729988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1837917-aa3f-4b3b-a6bd-08a31e28cb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:09:57.730589 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1837917-aa3f-4b3b-a6bd-08a31e28cb28 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:57.730864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1837917-aa3f-4b3b-a6bd-08a31e28cb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:57.731185 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1837917-aa3f-4b3b-a6bd-08a31e28cb28 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:57.731853 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1837917-aa3f-4b3b-a6bd-08a31e28cb28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:09:57.732869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: WARNING py.warnings [None req-e1837917-aa3f-4b3b-a6bd-08a31e28cb28 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 08:09:57.732869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: warnings.warn(msg) Oct 06 08:09:57.732869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: Oct 06 08:09:57.734677 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1837917-aa3f-4b3b-a6bd-08a31e28cb28 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:09:57.813424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 742/1498] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:09:57 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:09:57.817833 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59b329-2266-45f1-90d0-eaa7632bf3da admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:09:57.828263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59b329-2266-45f1-90d0-eaa7632bf3da admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:09:57.832937 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b48bbf-e5c7-407d-82c3-2d035fb25743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:09:57.833524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b48bbf-e5c7-407d-82c3-2d035fb25743 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:57.834845 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b48bbf-e5c7-407d-82c3-2d035fb25743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:57.834845 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b48bbf-e5c7-407d-82c3-2d035fb25743 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:57.834845 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b48bbf-e5c7-407d-82c3-2d035fb25743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:09:57.835697 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-17b48bbf-e5c7-407d-82c3-2d035fb25743 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 06 08:09:57.835697 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Oct 06 08:09:57.835697 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: Oct 06 08:09:57.836582 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b48bbf-e5c7-407d-82c3-2d035fb25743 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:09:57.849247 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 757/1499] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:09:57.853396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1837917-aa3f-4b3b-a6bd-08a31e28cb28 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:09:57.864341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1837917-aa3f-4b3b-a6bd-08a31e28cb28 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:09:57.868964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cefe14-b55a-4226-8dac-be3fe56ac06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:09:57.869587 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cefe14-b55a-4226-8dac-be3fe56ac06b admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:57.869864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cefe14-b55a-4226-8dac-be3fe56ac06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:57.870138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cefe14-b55a-4226-8dac-be3fe56ac06b admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:57.870771 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cefe14-b55a-4226-8dac-be3fe56ac06b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:09:57.871800 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: WARNING py.warnings [None req-72cefe14-b55a-4226-8dac-be3fe56ac06b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 06 08:09:57.871800 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: warnings.warn(msg) Oct 06 08:09:57.871800 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: Oct 06 08:09:57.872672 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cefe14-b55a-4226-8dac-be3fe56ac06b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:09:57.888013 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 743/1500] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:09:57.892856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b48bbf-e5c7-407d-82c3-2d035fb25743 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:09:57.903017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b48bbf-e5c7-407d-82c3-2d035fb25743 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:09:57.908159 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c7e32-1f0f-4bf0-abb0-02ef04f81154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:09:57.914106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c7e32-1f0f-4bf0-abb0-02ef04f81154 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:57.914106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c7e32-1f0f-4bf0-abb0-02ef04f81154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:57.914106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c7e32-1f0f-4bf0-abb0-02ef04f81154 admin admin] PATH_INFO: `/v3/projects/d27190873fb4497f999a0da80b184db8/users/73ae999e021e4228ba176762ecc3d42f/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:57.925267 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c7e32-1f0f-4bf0-abb0-02ef04f81154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d27190873fb4497f999a0da80b184db8, user_id=73ae999e021e4228ba176762ecc3d42f, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:09:57.926343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-810c7e32-1f0f-4bf0-abb0-02ef04f81154 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 08:09:57.926343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Oct 06 08:09:57.926343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: Oct 06 08:09:57.928327 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c7e32-1f0f-4bf0-abb0-02ef04f81154 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:09:58.002093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 758/1501] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:09:57 2022] PUT /identity/v3/projects/d27190873fb4497f999a0da80b184db8/users/73ae999e021e4228ba176762ecc3d42f/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:09:58.015125 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92155574-f29a-4756-82bc-2188a317ff0f None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:58.015968 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92155574-f29a-4756-82bc-2188a317ff0f None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:58.015968 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92155574-f29a-4756-82bc-2188a317ff0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:58.043429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-92155574-f29a-4756-82bc-2188a317ff0f None None] MFA Rules not processed for user `73ae999e021e4228ba176762ecc3d42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:09:58.044634 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-92155574-f29a-4756-82bc-2188a317ff0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:09:58.108134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 744/1502] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:09:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:09:58.122244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bb7ce-bf93-4fb4-a343-017eb216687c None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:58.122523 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bb7ce-bf93-4fb4-a343-017eb216687c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:58.126340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bb7ce-bf93-4fb4-a343-017eb216687c None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:58.126340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 759/1503] 10.222.0.30 () {54 vars in 963 bytes} [Thu Oct 6 08:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:09:58.129539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ebba9-ccf8-4419-99fa-ca6d87dad0be None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:58.129792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ebba9-ccf8-4419-99fa-ca6d87dad0be None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:58.130084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ebba9-ccf8-4419-99fa-ca6d87dad0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:58.196786 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-a59ebba9-ccf8-4419-99fa-ca6d87dad0be None None] MFA Rules not processed for user `a899c6dabca948269f6c1370425f244b`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:09:58.197986 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a59ebba9-ccf8-4419-99fa-ca6d87dad0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:09:58.271896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 745/1504] 10.222.0.30 () {58 vars in 1060 bytes} [Thu Oct 6 08:09:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:09:58.279673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bb7ce-bf93-4fb4-a343-017eb216687c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:09:58.290626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bb7ce-bf93-4fb4-a343-017eb216687c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:09:58.294078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa7040-c0ce-406f-be75-8946ae1baeb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:09:58.294656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa7040-c0ce-406f-be75-8946ae1baeb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:58.294928 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa7040-c0ce-406f-be75-8946ae1baeb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:58.295224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa7040-c0ce-406f-be75-8946ae1baeb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:58.304307 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa7040-c0ce-406f-be75-8946ae1baeb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:09:58.305786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa7040-c0ce-406f-be75-8946ae1baeb9 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:09:58.314530 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 760/1505] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:09:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:09:58.360049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a95de0-8fc1-42dd-b789-edf255a99795 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:58.360049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a95de0-8fc1-42dd-b789-edf255a99795 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:58.360362 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a95de0-8fc1-42dd-b789-edf255a99795 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:58.361552 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 746/1506] 10.222.0.30 () {52 vars in 830 bytes} [Thu Oct 6 08:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:09:58.365056 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa7040-c0ce-406f-be75-8946ae1baeb9 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:09:58.427574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa7040-c0ce-406f-be75-8946ae1baeb9 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:09:58.432128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa227a-39a4-4d5b-8247-5570ab138d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:09:58.432776 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa227a-39a4-4d5b-8247-5570ab138d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:58.433057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa227a-39a4-4d5b-8247-5570ab138d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:58.433324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa227a-39a4-4d5b-8247-5570ab138d2c admin admin] PATH_INFO: `/v3/users/73ae999e021e4228ba176762ecc3d42f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:58.434445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa227a-39a4-4d5b-8247-5570ab138d2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=73ae999e021e4228ba176762ecc3d42f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:09:58.435483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-0aaa227a-39a4-4d5b-8247-5570ab138d2c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 08:09:58.435483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Oct 06 08:09:58.435483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: Oct 06 08:09:58.436710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa227a-39a4-4d5b-8247-5570ab138d2c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:09:58.664883 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0aaa227a-39a4-4d5b-8247-5570ab138d2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73ae999e021e4228ba176762ecc3d42f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:09:58.688838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0aaa227a-39a4-4d5b-8247-5570ab138d2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73ae999e021e4228ba176762ecc3d42f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:09:58.689528 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0aaa227a-39a4-4d5b-8247-5570ab138d2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73ae999e021e4228ba176762ecc3d42f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:09:58.744725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0aaa227a-39a4-4d5b-8247-5570ab138d2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73ae999e021e4228ba176762ecc3d42f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:09:58.745541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0aaa227a-39a4-4d5b-8247-5570ab138d2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73ae999e021e4228ba176762ecc3d42f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:09:58.757288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:09:58.768053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 761/1507] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:09:58 2022] DELETE /identity/v3/users/73ae999e021e4228ba176762ecc3d42f => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:09:58.772539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd85d4-830c-455d-a474-fde56e947c33 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:58.772822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd85d4-830c-455d-a474-fde56e947c33 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:58.773098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd85d4-830c-455d-a474-fde56e947c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:58.792733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-06dd85d4-830c-455d-a474-fde56e947c33 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:09:58.793918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-06dd85d4-830c-455d-a474-fde56e947c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:09:58.803188 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:09:58.854222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 747/1508] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:09:58.864543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa227a-39a4-4d5b-8247-5570ab138d2c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:09:58.865960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0aaa227a-39a4-4d5b-8247-5570ab138d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:09:58.967334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa227a-39a4-4d5b-8247-5570ab138d2c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:09:58.970956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f84355-1d9e-49fb-b56a-6892bb7adb4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:09:58.971570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f84355-1d9e-49fb-b56a-6892bb7adb4a service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:58.971848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f84355-1d9e-49fb-b56a-6892bb7adb4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:58.972120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f84355-1d9e-49fb-b56a-6892bb7adb4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:58.981006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f84355-1d9e-49fb-b56a-6892bb7adb4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:09:58.982426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f84355-1d9e-49fb-b56a-6892bb7adb4a service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:09:58.991966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 762/1509] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:09:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:09:59.481787 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd85d4-830c-455d-a474-fde56e947c33 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:09:59.483402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-06dd85d4-830c-455d-a474-fde56e947c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:09:59.496682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd85d4-830c-455d-a474-fde56e947c33 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:09:59.502484 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f71eeb-7167-4824-a2d9-76a1eeb75bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:09:59.503061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f71eeb-7167-4824-a2d9-76a1eeb75bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:09:59.503340 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f71eeb-7167-4824-a2d9-76a1eeb75bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:09:59.503608 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f71eeb-7167-4824-a2d9-76a1eeb75bd1 admin admin] PATH_INFO: `/v3/projects/d27190873fb4497f999a0da80b184db8` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:09:59.505019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f71eeb-7167-4824-a2d9-76a1eeb75bd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d27190873fb4497f999a0da80b184db8)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:09:59.506041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: WARNING py.warnings [None req-12f71eeb-7167-4824-a2d9-76a1eeb75bd1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 08:09:59.506041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: warnings.warn(msg) Oct 06 08:09:59.506041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: Oct 06 08:09:59.507320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f71eeb-7167-4824-a2d9-76a1eeb75bd1 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:09:59.635598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-12f71eeb-7167-4824-a2d9-76a1eeb75bd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd27190873fb4497f999a0da80b184db8', 'request_id': 'req-12f71eeb-7167-4824-a2d9-76a1eeb75bd1'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:09:59.648942 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-12f71eeb-7167-4824-a2d9-76a1eeb75bd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd27190873fb4497f999a0da80b184db8', 'request_id': 'req-12f71eeb-7167-4824-a2d9-76a1eeb75bd1'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:09:59.736777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-12f71eeb-7167-4824-a2d9-76a1eeb75bd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd27190873fb4497f999a0da80b184db8', 'request_id': 'req-12f71eeb-7167-4824-a2d9-76a1eeb75bd1'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:09:59.747852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-12f71eeb-7167-4824-a2d9-76a1eeb75bd1 admin admin] The token cache is being invalidate because project d27190873fb4497f999a0da80b184db8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:09:59.748141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-12f71eeb-7167-4824-a2d9-76a1eeb75bd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:09:59.748904 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-12f71eeb-7167-4824-a2d9-76a1eeb75bd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:09:59.750394 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 748/1510] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:09:59 2022] DELETE /identity/v3/projects/d27190873fb4497f999a0da80b184db8 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:10:08.987800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=38 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:10:09.744797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:11:26.561118 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52581241-07b8-49e7-b95b-0046d066b634 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:26.561731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52581241-07b8-49e7-b95b-0046d066b634 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:26.561731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52581241-07b8-49e7-b95b-0046d066b634 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:26.563048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 08:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:11:26.567743 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09549a4-cb30-41f5-88de-4208d3574029 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:26.568139 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09549a4-cb30-41f5-88de-4208d3574029 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:26.568282 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09549a4-cb30-41f5-88de-4208d3574029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:26.589425 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-d09549a4-cb30-41f5-88de-4208d3574029 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:11:26.590618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d09549a4-cb30-41f5-88de-4208d3574029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:11:26.648728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 08:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:11:26.801180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52581241-07b8-49e7-b95b-0046d066b634 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:11:26.802921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-52581241-07b8-49e7-b95b-0046d066b634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:11:26.907718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52581241-07b8-49e7-b95b-0046d066b634 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:11:26.913073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9331091-38eb-42fc-9cbe-d29164761e3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:11:26.913673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9331091-38eb-42fc-9cbe-d29164761e3a service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:26.913950 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9331091-38eb-42fc-9cbe-d29164761e3a service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:26.914219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9331091-38eb-42fc-9cbe-d29164761e3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:26.922630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9331091-38eb-42fc-9cbe-d29164761e3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:11:26.924260 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9331091-38eb-42fc-9cbe-d29164761e3a service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:11:26.933651 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 764/1513] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:11:28.017179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887f519-82c7-42ac-b7cc-32c7d334ea24 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:28.017729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887f519-82c7-42ac-b7cc-32c7d334ea24 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:28.017729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887f519-82c7-42ac-b7cc-32c7d334ea24 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:28.019133 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 08:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:11:28.025240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4efbc-2f0e-498e-8b63-52e10db3e60c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:28.025691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4efbc-2f0e-498e-8b63-52e10db3e60c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:28.025813 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4efbc-2f0e-498e-8b63-52e10db3e60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:28.045098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dbe4efbc-2f0e-498e-8b63-52e10db3e60c None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:11:28.046299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dbe4efbc-2f0e-498e-8b63-52e10db3e60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:11:28.054133 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 08:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:11:28.205763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887f519-82c7-42ac-b7cc-32c7d334ea24 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:11:28.220207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887f519-82c7-42ac-b7cc-32c7d334ea24 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:11:28.225594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dae04-43bf-4e2b-b9d8-8f5f880102d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:11:28.226178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dae04-43bf-4e2b-b9d8-8f5f880102d7 service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:28.226459 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dae04-43bf-4e2b-b9d8-8f5f880102d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:28.226730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dae04-43bf-4e2b-b9d8-8f5f880102d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:28.235361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238dae04-43bf-4e2b-b9d8-8f5f880102d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:11:28.236951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238dae04-43bf-4e2b-b9d8-8f5f880102d7 service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:11:28.246870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 751/1516] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:11:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:11:28.588189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a5d90-0f74-4f8b-b31e-b1b1400c1af3 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:28.588400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a5d90-0f74-4f8b-b31e-b1b1400c1af3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:28.588624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a5d90-0f74-4f8b-b31e-b1b1400c1af3 None None] PATH_INFO: `/v3` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:28.589953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 766/1517] 10.222.0.43 () {54 vars in 925 bytes} [Thu Oct 6 08:11:28 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:11:28.595964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a5139-361d-4a40-90d8-b82f54ac5ca5 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:28.596251 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a5139-361d-4a40-90d8-b82f54ac5ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:28.596526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a5139-361d-4a40-90d8-b82f54ac5ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:28.654167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-064a5139-361d-4a40-90d8-b82f54ac5ca5 None None] MFA Rules not processed for user `6074872303d3468fb566378fa34e8bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:11:28.655430 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-064a5139-361d-4a40-90d8-b82f54ac5ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:11:28.726779 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 752/1518] 10.222.0.43 () {58 vars in 1015 bytes} [Thu Oct 6 08:11:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:11:37.740857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4b417-6265-4df7-89ab-d7bbd3504213 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:37.741320 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4b417-6265-4df7-89ab-d7bbd3504213 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:37.741435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4b417-6265-4df7-89ab-d7bbd3504213 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:37.743339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 767/1519] 10.222.0.30 () {54 vars in 961 bytes} [Thu Oct 6 08:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:11:37.748694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0544f-d28c-4596-a7b0-ac751fa06402 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:37.749127 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0544f-d28c-4596-a7b0-ac751fa06402 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:37.749226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0544f-d28c-4596-a7b0-ac751fa06402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:37.769349 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-e7c0544f-d28c-4596-a7b0-ac751fa06402 None None] MFA Rules not processed for user `6074872303d3468fb566378fa34e8bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:11:37.770470 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e7c0544f-d28c-4596-a7b0-ac751fa06402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:11:37.779114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 753/1520] 10.222.0.30 () {58 vars in 1058 bytes} [Thu Oct 6 08:11:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:11:37.786744 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4b417-6265-4df7-89ab-d7bbd3504213 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:11:37.799777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4b417-6265-4df7-89ab-d7bbd3504213 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:11:37.805580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe01f0-b679-432d-ae41-659e543de05e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6074872303d3468fb566378fa34e8bea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:11:37.809983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe01f0-b679-432d-ae41-659e543de05e service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:37.809983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe01f0-b679-432d-ae41-659e543de05e service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:37.809983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe01f0-b679-432d-ae41-659e543de05e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:37.814697 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe01f0-b679-432d-ae41-659e543de05e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:11:37.816292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe01f0-b679-432d-ae41-659e543de05e service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:11:37.826493 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 768/1521] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:11:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:11:39.760179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba04ac-03a8-4f50-bd13-f7f0a040dd42 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:39.760713 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba04ac-03a8-4f50-bd13-f7f0a040dd42 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:39.760713 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba04ac-03a8-4f50-bd13-f7f0a040dd42 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:39.762081 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 08:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:11:39.768330 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0b7ad-6e51-4650-abe4-4f6c40004dd2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:39.768749 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0b7ad-6e51-4650-abe4-4f6c40004dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:39.768854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0b7ad-6e51-4650-abe4-4f6c40004dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:39.789347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b4f0b7ad-6e51-4650-abe4-4f6c40004dd2 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:11:39.790541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b4f0b7ad-6e51-4650-abe4-4f6c40004dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:11:39.798758 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 08:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:11:39.953874 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba04ac-03a8-4f50-bd13-f7f0a040dd42 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:11:39.968249 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba04ac-03a8-4f50-bd13-f7f0a040dd42 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:11:39.973689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9bfa6-7508-46f2-83e0-c80b7a8281ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:11:39.974278 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9bfa6-7508-46f2-83e0-c80b7a8281ba service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:39.974554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9bfa6-7508-46f2-83e0-c80b7a8281ba service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:39.974824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9bfa6-7508-46f2-83e0-c80b7a8281ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:39.983091 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9bfa6-7508-46f2-83e0-c80b7a8281ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:11:39.984677 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9bfa6-7508-46f2-83e0-c80b7a8281ba service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:11:39.994914 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 755/1524] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:11:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:11:41.091738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd0179-e33c-432b-b58e-c103a60849bb None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:41.092311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd0179-e33c-432b-b58e-c103a60849bb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:41.092311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd0179-e33c-432b-b58e-c103a60849bb None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:41.093644 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 6 08:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 08:11:41.100191 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c961c4-bc60-431f-9054-f0fa7888462f None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:41.100809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c961c4-bc60-431f-9054-f0fa7888462f None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:41.100809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c961c4-bc60-431f-9054-f0fa7888462f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:41.119833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-10c961c4-bc60-431f-9054-f0fa7888462f None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:11:41.121240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-10c961c4-bc60-431f-9054-f0fa7888462f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:11:41.129254 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 6 08:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:11:41.280377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd0179-e33c-432b-b58e-c103a60849bb None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:11:41.294724 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd0179-e33c-432b-b58e-c103a60849bb None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:11:41.300099 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3b53c-8bce-4aa0-84ea-a0b03a5988e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:11:41.300688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3b53c-8bce-4aa0-84ea-a0b03a5988e1 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:41.300960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3b53c-8bce-4aa0-84ea-a0b03a5988e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:41.301226 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3b53c-8bce-4aa0-84ea-a0b03a5988e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:41.309750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3b53c-8bce-4aa0-84ea-a0b03a5988e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:11:41.311352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3b53c-8bce-4aa0-84ea-a0b03a5988e1 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:11:41.320907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 771/1527] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:11:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:11:45.355113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3b01a-6c35-41ea-99bd-5dbdd47496fd None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:45.355705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3b01a-6c35-41ea-99bd-5dbdd47496fd None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:45.355705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3b01a-6c35-41ea-99bd-5dbdd47496fd None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:45.357084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 08:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:11:45.362411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc265500-1041-42c6-b188-3d3c6261db1e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:45.362836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc265500-1041-42c6-b188-3d3c6261db1e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:45.362974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc265500-1041-42c6-b188-3d3c6261db1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:45.382227 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cc265500-1041-42c6-b188-3d3c6261db1e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:11:45.383611 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cc265500-1041-42c6-b188-3d3c6261db1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:11:45.391309 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 08:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:11:45.680373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3b01a-6c35-41ea-99bd-5dbdd47496fd None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:11:45.682107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-90a3b01a-6c35-41ea-99bd-5dbdd47496fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:11:45.710051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:11:45.739415 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3b01a-6c35-41ea-99bd-5dbdd47496fd None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:11:45.744046 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2512cc-952e-45e7-b93a-9d1cf1592a94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:11:45.744658 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2512cc-952e-45e7-b93a-9d1cf1592a94 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:45.744936 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2512cc-952e-45e7-b93a-9d1cf1592a94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:45.745209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2512cc-952e-45e7-b93a-9d1cf1592a94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:45.754440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2512cc-952e-45e7-b93a-9d1cf1592a94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:11:45.755909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2512cc-952e-45e7-b93a-9d1cf1592a94 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:11:45.766332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 758/1530] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:11:47.985539 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd44da0-3ef7-4799-a3a4-78ca3bdfaf09 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:47.986131 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd44da0-3ef7-4799-a3a4-78ca3bdfaf09 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:47.986131 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd44da0-3ef7-4799-a3a4-78ca3bdfaf09 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:47.987535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 08:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:11:47.993451 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ea51b-3648-41aa-b0e9-12b5c44b2494 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:47.993877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ea51b-3648-41aa-b0e9-12b5c44b2494 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:47.993985 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ea51b-3648-41aa-b0e9-12b5c44b2494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:48.014395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-8e2ea51b-3648-41aa-b0e9-12b5c44b2494 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:11:48.015612 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-8e2ea51b-3648-41aa-b0e9-12b5c44b2494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:11:48.024299 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 08:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:11:48.314019 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd44da0-3ef7-4799-a3a4-78ca3bdfaf09 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:11:48.315738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dfd44da0-3ef7-4799-a3a4-78ca3bdfaf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:11:48.329292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd44da0-3ef7-4799-a3a4-78ca3bdfaf09 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:11:48.332736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc992d4-ba78-4879-93e4-e45b0eee0ef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:11:48.333321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc992d4-ba78-4879-93e4-e45b0eee0ef7 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:48.333606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc992d4-ba78-4879-93e4-e45b0eee0ef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:48.333854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc992d4-ba78-4879-93e4-e45b0eee0ef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:48.342995 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc992d4-ba78-4879-93e4-e45b0eee0ef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:11:48.344411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc992d4-ba78-4879-93e4-e45b0eee0ef7 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:11:48.354102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 774/1533] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:11:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:11:51.986479 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305200f-e51d-43a1-9c4b-68b558502913 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:51.987022 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305200f-e51d-43a1-9c4b-68b558502913 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:51.987159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305200f-e51d-43a1-9c4b-68b558502913 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:51.988453 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 08:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:11:51.993661 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd98423-726f-44d2-8d65-f6da29b0d181 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:51.994090 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd98423-726f-44d2-8d65-f6da29b0d181 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:51.994199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd98423-726f-44d2-8d65-f6da29b0d181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:52.001756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=74 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:11:52.014211 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0bd98423-726f-44d2-8d65-f6da29b0d181 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:11:52.015427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0bd98423-726f-44d2-8d65-f6da29b0d181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:11:52.023627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 08:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:11:52.325391 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305200f-e51d-43a1-9c4b-68b558502913 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:11:52.327089 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a305200f-e51d-43a1-9c4b-68b558502913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:11:52.384010 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305200f-e51d-43a1-9c4b-68b558502913 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:11:52.387517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f14ca-6dae-42f1-8319-eb7232b2bca8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:11:52.388104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f14ca-6dae-42f1-8319-eb7232b2bca8 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:52.388376 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f14ca-6dae-42f1-8319-eb7232b2bca8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:52.388642 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f14ca-6dae-42f1-8319-eb7232b2bca8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:52.397350 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f14ca-6dae-42f1-8319-eb7232b2bca8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:11:52.399401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f14ca-6dae-42f1-8319-eb7232b2bca8 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:11:52.408548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 761/1536] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:11:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:11:54.581057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de047d3-58ba-411a-b343-8c987955dcd0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:54.581629 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de047d3-58ba-411a-b343-8c987955dcd0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:54.581629 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de047d3-58ba-411a-b343-8c987955dcd0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:54.583004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 6 08:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:11:54.588879 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1cf4cf-edf7-4fbd-a44f-0f5220cc784f None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:54.589273 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1cf4cf-edf7-4fbd-a44f-0f5220cc784f None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:54.589422 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1cf4cf-edf7-4fbd-a44f-0f5220cc784f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:54.608743 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-af1cf4cf-edf7-4fbd-a44f-0f5220cc784f None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:11:54.610119 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-af1cf4cf-edf7-4fbd-a44f-0f5220cc784f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:11:54.617990 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 6 08:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:11:54.917633 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de047d3-58ba-411a-b343-8c987955dcd0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:11:54.919326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4de047d3-58ba-411a-b343-8c987955dcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:11:54.932718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de047d3-58ba-411a-b343-8c987955dcd0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:11:54.936210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854235b5-410c-4002-99c2-a38c4b811445 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:11:54.936794 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854235b5-410c-4002-99c2-a38c4b811445 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:11:54.937068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854235b5-410c-4002-99c2-a38c4b811445 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:11:54.937335 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854235b5-410c-4002-99c2-a38c4b811445 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:11:54.946771 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854235b5-410c-4002-99c2-a38c4b811445 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:11:54.948258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854235b5-410c-4002-99c2-a38c4b811445 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:11:54.957870 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 777/1539] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:11:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:12:04.603362 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=47 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:12:04.953591 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=8 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:19.233509 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c87b8-aec7-407c-87fd-81ef9fe530e5 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.234159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c87b8-aec7-407c-87fd-81ef9fe530e5 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.234159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c87b8-aec7-407c-87fd-81ef9fe530e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.260067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-295c87b8-aec7-407c-87fd-81ef9fe530e5 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:19.261696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-295c87b8-aec7-407c-87fd-81ef9fe530e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:19.275863 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 763/1540] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:19.286331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e74190-210f-4ccb-85b2-ac7f81dc9c09 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.286988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e74190-210f-4ccb-85b2-ac7f81dc9c09 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.286988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e74190-210f-4ccb-85b2-ac7f81dc9c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.317098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-53e74190-210f-4ccb-85b2-ac7f81dc9c09 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:19.327225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-53e74190-210f-4ccb-85b2-ac7f81dc9c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:19.335936 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c5c5b-c261-4790-80e4-2d2674242468 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.336882 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c5c5b-c261-4790-80e4-2d2674242468 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.337077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 778/1541] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:19.337693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c5c5b-c261-4790-80e4-2d2674242468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.350657 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e74190-210f-4ccb-85b2-ac7f81dc9c09 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:19.369454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e74190-210f-4ccb-85b2-ac7f81dc9c09 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:19.375124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-7b0c5c5b-c261-4790-80e4-2d2674242468 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:19.376896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7b0c5c5b-c261-4790-80e4-2d2674242468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:19.381001 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf0c17-0af0-4468-9499-9fd57b5e056c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:19.381688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf0c17-0af0-4468-9499-9fd57b5e056c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.381920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf0c17-0af0-4468-9499-9fd57b5e056c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.382171 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf0c17-0af0-4468-9499-9fd57b5e056c admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.382904 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf0c17-0af0-4468-9499-9fd57b5e056c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:13:19.383240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf0c17-0af0-4468-9499-9fd57b5e056c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:19.384284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-badf0c17-0af0-4468-9499-9fd57b5e056c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 06 08:13:19.384284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Oct 06 08:13:19.384284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: Oct 06 08:13:19.385289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf0c17-0af0-4468-9499-9fd57b5e056c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:19.394180 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 764/1542] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 06 08:13:19.404106 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0be6b-1ad5-4a67-9ea9-f58234e08946 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.404730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0be6b-1ad5-4a67-9ea9-f58234e08946 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.405581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 779/1543] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 08:13:19.406095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0be6b-1ad5-4a67-9ea9-f58234e08946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.425965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf0c17-0af0-4468-9499-9fd57b5e056c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:19.441933 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf0c17-0af0-4468-9499-9fd57b5e056c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:19.443345 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-e6a0be6b-1ad5-4a67-9ea9-f58234e08946 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:19.445155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e6a0be6b-1ad5-4a67-9ea9-f58234e08946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:19.456531 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee7907-55b5-49fd-a2f6-9f544ba13b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:19.457166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee7907-55b5-49fd-a2f6-9f544ba13b54 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.457438 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee7907-55b5-49fd-a2f6-9f544ba13b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.457705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee7907-55b5-49fd-a2f6-9f544ba13b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.458474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee7907-55b5-49fd-a2f6-9f544ba13b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:19.462965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee7907-55b5-49fd-a2f6-9f544ba13b54 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:19.469954 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 765/1544] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:13:19.474107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0be6b-1ad5-4a67-9ea9-f58234e08946 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:19.490075 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0be6b-1ad5-4a67-9ea9-f58234e08946 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:19.501993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5496122-bef3-4be4-8668-465ce6eed3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:19.503773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496122-bef3-4be4-8668-465ce6eed3cf admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.503773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496122-bef3-4be4-8668-465ce6eed3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.503773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496122-bef3-4be4-8668-465ce6eed3cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.504164 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5496122-bef3-4be4-8668-465ce6eed3cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:13:19.504389 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5496122-bef3-4be4-8668-465ce6eed3cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:19.506106 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5496122-bef3-4be4-8668-465ce6eed3cf admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:19.528448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 766/1545] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 08:13:19.536933 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5496122-bef3-4be4-8668-465ce6eed3cf admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:19.552318 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5496122-bef3-4be4-8668-465ce6eed3cf admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:19.560061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6811b3f-ef93-4895-a370-f0c0d4ea6d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:19.560677 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6811b3f-ef93-4895-a370-f0c0d4ea6d08 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.560954 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6811b3f-ef93-4895-a370-f0c0d4ea6d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.561226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6811b3f-ef93-4895-a370-f0c0d4ea6d08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.561865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6811b3f-ef93-4895-a370-f0c0d4ea6d08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:19.562895 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: WARNING py.warnings [None req-c6811b3f-ef93-4895-a370-f0c0d4ea6d08 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 08:13:19.562895 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: warnings.warn(msg) Oct 06 08:13:19.562895 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: Oct 06 08:13:19.564191 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6811b3f-ef93-4895-a370-f0c0d4ea6d08 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:19.615161 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 780/1546] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:13:19 2022] POST /identity/v3/projects => generated 373 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:19.616439 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac854e7-94de-461d-a52e-469734d62243 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.616730 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac854e7-94de-461d-a52e-469734d62243 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.616977 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac854e7-94de-461d-a52e-469734d62243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.647645 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 767/1547] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:13:19 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:13:19.648225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4ac854e7-94de-461d-a52e-469734d62243 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:19.648488 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6811b3f-ef93-4895-a370-f0c0d4ea6d08 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:19.649316 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4ac854e7-94de-461d-a52e-469734d62243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:19.746824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 781/1548] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:19.749643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6811b3f-ef93-4895-a370-f0c0d4ea6d08 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:19.750554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac854e7-94de-461d-a52e-469734d62243 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:19.754852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3acdc-01f3-4bf8-a665-bfd65c1e5a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:19.755519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3acdc-01f3-4bf8-a665-bfd65c1e5a35 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.755798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3acdc-01f3-4bf8-a665-bfd65c1e5a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.756065 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3acdc-01f3-4bf8-a665-bfd65c1e5a35 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.756717 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3acdc-01f3-4bf8-a665-bfd65c1e5a35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:19.758710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3acdc-01f3-4bf8-a665-bfd65c1e5a35 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:19.765067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac854e7-94de-461d-a52e-469734d62243 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:19.770115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec8272-98d8-4284-945a-9d70cb105b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:19.770616 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec8272-98d8-4284-945a-9d70cb105b0a admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.770741 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec8272-98d8-4284-945a-9d70cb105b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.771098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec8272-98d8-4284-945a-9d70cb105b0a admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.771708 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec8272-98d8-4284-945a-9d70cb105b0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:19.772720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-eaec8272-98d8-4284-945a-9d70cb105b0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 08:13:19.772720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Oct 06 08:13:19.772720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: Oct 06 08:13:19.773939 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec8272-98d8-4284-945a-9d70cb105b0a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:19.876433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 768/1549] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:13:19 2022] POST /identity/v3/users => generated 287 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:19.877678 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b9744-5257-4a2c-b089-654cd815180b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.877961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b9744-5257-4a2c-b089-654cd815180b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.878232 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b9744-5257-4a2c-b089-654cd815180b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.887970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 782/1550] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:13:19 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:19.889889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc4fe4-9b69-45d5-b6c4-c0f14dec6ede None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.890168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc4fe4-9b69-45d5-b6c4-c0f14dec6ede None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.890534 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc4fe4-9b69-45d5-b6c4-c0f14dec6ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.898782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-494b9744-5257-4a2c-b089-654cd815180b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:19.900161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-494b9744-5257-4a2c-b089-654cd815180b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:19.909058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 769/1551] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:19.910264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8c7e9-5fcc-407b-b265-3a3af63e8d7d None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.910543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8c7e9-5fcc-407b-b265-3a3af63e8d7d None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.910812 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8c7e9-5fcc-407b-b265-3a3af63e8d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.912081 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-33fc4fe4-9b69-45d5-b6c4-c0f14dec6ede None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:19.913363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-33fc4fe4-9b69-45d5-b6c4-c0f14dec6ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:19.921750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 783/1552] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:19.923064 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f64622-bc66-4a26-bcca-0d55262a98d4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.923343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f64622-bc66-4a26-bcca-0d55262a98d4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.923614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f64622-bc66-4a26-bcca-0d55262a98d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.935631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-5ef8c7e9-5fcc-407b-b265-3a3af63e8d7d None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:19.936838 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5ef8c7e9-5fcc-407b-b265-3a3af63e8d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:19.943101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d7f64622-bc66-4a26-bcca-0d55262a98d4 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:19.944249 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d7f64622-bc66-4a26-bcca-0d55262a98d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:19.945189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 770/1553] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:19.952732 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b024a4-4a13-45cd-9351-8d89d6460ade None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.952875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 784/1554] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:19.954513 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b024a4-4a13-45cd-9351-8d89d6460ade None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.954513 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b024a4-4a13-45cd-9351-8d89d6460ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.956209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37625d60-a0e1-4dc8-a894-1100cf63be98 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:19.956584 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37625d60-a0e1-4dc8-a894-1100cf63be98 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:19.956962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37625d60-a0e1-4dc8-a894-1100cf63be98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:19.973666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-a5b024a4-4a13-45cd-9351-8d89d6460ade None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:19.975545 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a5b024a4-4a13-45cd-9351-8d89d6460ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:19.978860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-37625d60-a0e1-4dc8-a894-1100cf63be98 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:19.980042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-37625d60-a0e1-4dc8-a894-1100cf63be98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:19.983679 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 771/1555] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:19.984437 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b024a4-4a13-45cd-9351-8d89d6460ade None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:19.991604 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 785/1556] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:19.992257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37625d60-a0e1-4dc8-a894-1100cf63be98 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:19.994141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b024a4-4a13-45cd-9351-8d89d6460ade None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:19.999414 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9de780-9811-4df0-b1cd-2c02d4dc64c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:19.999954 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9de780-9811-4df0-b1cd-2c02d4dc64c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.000233 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9de780-9811-4df0-b1cd-2c02d4dc64c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.000530 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9de780-9811-4df0-b1cd-2c02d4dc64c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.001170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9de780-9811-4df0-b1cd-2c02d4dc64c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.001539 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37625d60-a0e1-4dc8-a894-1100cf63be98 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.003726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9de780-9811-4df0-b1cd-2c02d4dc64c9 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.006516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f08d8-a50f-43a1-b927-734fe5de5c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.007138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f08d8-a50f-43a1-b927-734fe5de5c82 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.007415 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f08d8-a50f-43a1-b927-734fe5de5c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.007682 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f08d8-a50f-43a1-b927-734fe5de5c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.008325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f08d8-a50f-43a1-b927-734fe5de5c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.009912 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f08d8-a50f-43a1-b927-734fe5de5c82 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.017483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 772/1557] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:20.017483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49917b-cf2d-4690-ac0a-3abfcedf0133 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.017483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49917b-cf2d-4690-ac0a-3abfcedf0133 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.017483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49917b-cf2d-4690-ac0a-3abfcedf0133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.025795 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 786/1558] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:20.025795 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb68be-99a7-4235-a4ab-a7360bdc10c8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.025795 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb68be-99a7-4235-a4ab-a7360bdc10c8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.025795 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb68be-99a7-4235-a4ab-a7360bdc10c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.037117 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-8f49917b-cf2d-4690-ac0a-3abfcedf0133 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:20.038318 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-8f49917b-cf2d-4690-ac0a-3abfcedf0133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:20.045017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a3eb68be-99a7-4235-a4ab-a7360bdc10c8 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:20.046198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a3eb68be-99a7-4235-a4ab-a7360bdc10c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:20.047516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 773/1559] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:20.048562 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a82f5-767d-4cb0-a7f5-04e5a13e022b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.048839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a82f5-767d-4cb0-a7f5-04e5a13e022b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.049108 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a82f5-767d-4cb0-a7f5-04e5a13e022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.054314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 787/1560] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:20.055618 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbcf29-28b8-4eda-80d7-f7a1260f753d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.055894 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbcf29-28b8-4eda-80d7-f7a1260f753d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.056161 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbcf29-28b8-4eda-80d7-f7a1260f753d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.069560 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-f34a82f5-767d-4cb0-a7f5-04e5a13e022b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:20.070747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f34a82f5-767d-4cb0-a7f5-04e5a13e022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:20.076056 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9dfbcf29-28b8-4eda-80d7-f7a1260f753d None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:20.077213 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9dfbcf29-28b8-4eda-80d7-f7a1260f753d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:20.079798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 774/1561] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:20.080650 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a82f5-767d-4cb0-a7f5-04e5a13e022b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:20.085349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 788/1562] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:20.087190 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff92c9-5451-4558-a00d-5343c3a6320b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.087483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff92c9-5451-4558-a00d-5343c3a6320b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.087754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff92c9-5451-4558-a00d-5343c3a6320b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.090917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a82f5-767d-4cb0-a7f5-04e5a13e022b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.095630 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaaec3b-2a67-4e98-ae2a-1bcb877b6059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.096252 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaec3b-2a67-4e98-ae2a-1bcb877b6059 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.096523 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaec3b-2a67-4e98-ae2a-1bcb877b6059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.096790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaec3b-2a67-4e98-ae2a-1bcb877b6059 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.097433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaaec3b-2a67-4e98-ae2a-1bcb877b6059 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:13:20.097729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaaec3b-2a67-4e98-ae2a-1bcb877b6059 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.099492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaaec3b-2a67-4e98-ae2a-1bcb877b6059 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.109201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-82ff92c9-5451-4558-a00d-5343c3a6320b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:20.110414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-82ff92c9-5451-4558-a00d-5343c3a6320b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:20.111842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 775/1563] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:13:20.117242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaaec3b-2a67-4e98-ae2a-1bcb877b6059 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:20.119392 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 789/1564] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:20.120169 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff92c9-5451-4558-a00d-5343c3a6320b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:20.129242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaaec3b-2a67-4e98-ae2a-1bcb877b6059 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.130913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff92c9-5451-4558-a00d-5343c3a6320b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.134175 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fa971-6253-4033-90ed-71b676f1dcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.134768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fa971-6253-4033-90ed-71b676f1dcec admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.135077 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fa971-6253-4033-90ed-71b676f1dcec admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.135362 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fa971-6253-4033-90ed-71b676f1dcec admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.135808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f8576-2c44-4eca-b427-7b56bf64e751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.136073 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347fa971-6253-4033-90ed-71b676f1dcec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.136435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f8576-2c44-4eca-b427-7b56bf64e751 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.137391 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f8576-2c44-4eca-b427-7b56bf64e751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.137662 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f8576-2c44-4eca-b427-7b56bf64e751 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.138303 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347fa971-6253-4033-90ed-71b676f1dcec admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.138480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f8576-2c44-4eca-b427-7b56bf64e751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.140728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f8576-2c44-4eca-b427-7b56bf64e751 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.152403 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 776/1565] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:20.152403 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fa971-6253-4033-90ed-71b676f1dcec admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:20.155615 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 790/1566] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:20.156346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f8576-2c44-4eca-b427-7b56bf64e751 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:20.162238 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fa971-6253-4033-90ed-71b676f1dcec admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.167183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f8576-2c44-4eca-b427-7b56bf64e751 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.167908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858e8f5-a7d9-4812-9695-841edb16b163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.168854 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e8f5-a7d9-4812-9695-841edb16b163 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.169315 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e8f5-a7d9-4812-9695-841edb16b163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.169594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e8f5-a7d9-4812-9695-841edb16b163 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.170239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858e8f5-a7d9-4812-9695-841edb16b163 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:13:20.170538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858e8f5-a7d9-4812-9695-841edb16b163 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.171932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19494840-fbf8-48c0-a584-cdf003c6331d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.172299 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858e8f5-a7d9-4812-9695-841edb16b163 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.172524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19494840-fbf8-48c0-a584-cdf003c6331d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.172797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19494840-fbf8-48c0-a584-cdf003c6331d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.173066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19494840-fbf8-48c0-a584-cdf003c6331d admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.173705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19494840-fbf8-48c0-a584-cdf003c6331d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:13:20.174006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19494840-fbf8-48c0-a584-cdf003c6331d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.175627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19494840-fbf8-48c0-a584-cdf003c6331d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.184913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 777/1567] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:13:20.186220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858e8f5-a7d9-4812-9695-841edb16b163 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:20.189650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 791/1568] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:13:20.190481 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19494840-fbf8-48c0-a584-cdf003c6331d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:20.201238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19494840-fbf8-48c0-a584-cdf003c6331d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.202729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858e8f5-a7d9-4812-9695-841edb16b163 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.206116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed1550-c436-44f3-882a-ec46a15dd7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.206713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed1550-c436-44f3-882a-ec46a15dd7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.211162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed1550-c436-44f3-882a-ec46a15dd7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.211162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed1550-c436-44f3-882a-ec46a15dd7e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.211438 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe1345-8041-4628-9868-e5123a82ce15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.211539 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed1550-c436-44f3-882a-ec46a15dd7e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:13:20.211633 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe1345-8041-4628-9868-e5123a82ce15 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.211719 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed1550-c436-44f3-882a-ec46a15dd7e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.211794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe1345-8041-4628-9868-e5123a82ce15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.211794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe1345-8041-4628-9868-e5123a82ce15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.211794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe1345-8041-4628-9868-e5123a82ce15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:13:20.211794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe1345-8041-4628-9868-e5123a82ce15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.212072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed1550-c436-44f3-882a-ec46a15dd7e6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.212156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe1345-8041-4628-9868-e5123a82ce15 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.220771 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:20.221987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 792/1569] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:13:20.223057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed1550-c436-44f3-882a-ec46a15dd7e6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:20.230916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 778/1570] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:13:20.230916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe1345-8041-4628-9868-e5123a82ce15 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:20.236517 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed1550-c436-44f3-882a-ec46a15dd7e6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.240342 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe1345-8041-4628-9868-e5123a82ce15 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.241695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d11a5-4c0a-49f9-b027-3bcc284b0e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.242299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d11a5-4c0a-49f9-b027-3bcc284b0e36 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.242576 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d11a5-4c0a-49f9-b027-3bcc284b0e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.242847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d11a5-4c0a-49f9-b027-3bcc284b0e36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.243549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d11a5-4c0a-49f9-b027-3bcc284b0e36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:13:20.243850 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d11a5-4c0a-49f9-b027-3bcc284b0e36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.244872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262bbe21-8178-48a5-b561-bdd55ff2c0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.245448 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d11a5-4c0a-49f9-b027-3bcc284b0e36 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.245558 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bbe21-8178-48a5-b561-bdd55ff2c0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.245761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bbe21-8178-48a5-b561-bdd55ff2c0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.246548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bbe21-8178-48a5-b561-bdd55ff2c0f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.247274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262bbe21-8178-48a5-b561-bdd55ff2c0f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.249168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262bbe21-8178-48a5-b561-bdd55ff2c0f8 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.260053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:20.260053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 793/1571] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:13:20.260053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d11a5-4c0a-49f9-b027-3bcc284b0e36 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:20.269999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d11a5-4c0a-49f9-b027-3bcc284b0e36 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.274734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298109d-78d7-4f1d-b6f1-bc37c0a96dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.275420 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298109d-78d7-4f1d-b6f1-bc37c0a96dff admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.275693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298109d-78d7-4f1d-b6f1-bc37c0a96dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.275964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298109d-78d7-4f1d-b6f1-bc37c0a96dff admin admin] PATH_INFO: `/v3/projects/4a1c57aa8b7f4dd0bcf397ff84097284/users/5fb488cb6e2d443db89acfb629501d82/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.290315 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298109d-78d7-4f1d-b6f1-bc37c0a96dff admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a1c57aa8b7f4dd0bcf397ff84097284, user_id=5fb488cb6e2d443db89acfb629501d82, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.293044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298109d-78d7-4f1d-b6f1-bc37c0a96dff admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.355543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 779/1572] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:13:20 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:20.356402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262bbe21-8178-48a5-b561-bdd55ff2c0f8 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:20.379096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 794/1573] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:13:20 2022] PUT /identity/v3/projects/4a1c57aa8b7f4dd0bcf397ff84097284/users/5fb488cb6e2d443db89acfb629501d82/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:20.386182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298109d-78d7-4f1d-b6f1-bc37c0a96dff admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:20.425714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262bbe21-8178-48a5-b561-bdd55ff2c0f8 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.449857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298109d-78d7-4f1d-b6f1-bc37c0a96dff admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.454712 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aff729-5507-4663-a05b-b2bea56dee5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.458089 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aff729-5507-4663-a05b-b2bea56dee5d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.458089 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aff729-5507-4663-a05b-b2bea56dee5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.458089 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aff729-5507-4663-a05b-b2bea56dee5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.458089 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aff729-5507-4663-a05b-b2bea56dee5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.458454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aff729-5507-4663-a05b-b2bea56dee5d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.483226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e713b8-67a6-41cc-948f-d6da28ae7538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.483917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e713b8-67a6-41cc-948f-d6da28ae7538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.484151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e713b8-67a6-41cc-948f-d6da28ae7538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.484419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e713b8-67a6-41cc-948f-d6da28ae7538 admin admin] PATH_INFO: `/v3/projects/546cd66807054a5a8d7a8137e90a4f36/users/3907dcef06814ce7a1434d9c451c1980/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.499259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e713b8-67a6-41cc-948f-d6da28ae7538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=546cd66807054a5a8d7a8137e90a4f36, user_id=3907dcef06814ce7a1434d9c451c1980, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.500287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: WARNING py.warnings [None req-65e713b8-67a6-41cc-948f-d6da28ae7538 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 08:13:20.500287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: warnings.warn(msg) Oct 06 08:13:20.500287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: Oct 06 08:13:20.502208 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e713b8-67a6-41cc-948f-d6da28ae7538 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.582217 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 795/1574] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:13:20 2022] POST /identity/v3/projects => generated 377 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:20.582217 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aff729-5507-4663-a05b-b2bea56dee5d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:20.618228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 780/1575] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:13:20 2022] PUT /identity/v3/projects/546cd66807054a5a8d7a8137e90a4f36/users/3907dcef06814ce7a1434d9c451c1980/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:20.619063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e713b8-67a6-41cc-948f-d6da28ae7538 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:20.643650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aff729-5507-4663-a05b-b2bea56dee5d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.688228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e713b8-67a6-41cc-948f-d6da28ae7538 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.692912 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a0b99-eac5-4c30-9a3b-2c992952582b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.693546 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a0b99-eac5-4c30-9a3b-2c992952582b admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.693825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a0b99-eac5-4c30-9a3b-2c992952582b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.694096 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a0b99-eac5-4c30-9a3b-2c992952582b admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.694719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a0b99-eac5-4c30-9a3b-2c992952582b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.696700 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a0b99-eac5-4c30-9a3b-2c992952582b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.699927 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b0aed-18b1-4dee-843e-6c8cc0b78c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.700531 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b0aed-18b1-4dee-843e-6c8cc0b78c75 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.700806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b0aed-18b1-4dee-843e-6c8cc0b78c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.701199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b0aed-18b1-4dee-843e-6c8cc0b78c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.701829 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b0aed-18b1-4dee-843e-6c8cc0b78c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.703822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b0aed-18b1-4dee-843e-6c8cc0b78c75 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.711199 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:20.718080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:20.810590 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 781/1576] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:13:20 2022] POST /identity/v3/projects => generated 379 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:20.811664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a0b99-eac5-4c30-9a3b-2c992952582b admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:20.825890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 796/1577] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:13:20 2022] POST /identity/v3/projects => generated 365 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:20.826828 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b0aed-18b1-4dee-843e-6c8cc0b78c75 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:20.874320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a0b99-eac5-4c30-9a3b-2c992952582b admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.889781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b0aed-18b1-4dee-843e-6c8cc0b78c75 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:20.894428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50783e-fe97-4a60-a85b-e214c49d34e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.895044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50783e-fe97-4a60-a85b-e214c49d34e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.895339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50783e-fe97-4a60-a85b-e214c49d34e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.895611 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50783e-fe97-4a60-a85b-e214c49d34e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.896235 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50783e-fe97-4a60-a85b-e214c49d34e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.898119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50783e-fe97-4a60-a85b-e214c49d34e3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:20.931132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7b971-d188-4743-9e1d-fec98c2f1b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:20.931736 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7b971-d188-4743-9e1d-fec98c2f1b3d admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:20.932011 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7b971-d188-4743-9e1d-fec98c2f1b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:20.932280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7b971-d188-4743-9e1d-fec98c2f1b3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:20.932903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7b971-d188-4743-9e1d-fec98c2f1b3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:20.935003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7b971-d188-4743-9e1d-fec98c2f1b3d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.032284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 797/1578] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:13:20 2022] POST /identity/v3/projects => generated 383 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:21.033109 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50783e-fe97-4a60-a85b-e214c49d34e3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.071975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 782/1579] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:13:20 2022] POST /identity/v3/projects => generated 393 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:21.073253 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0e12f-c9f2-403f-b5bd-3ab39768e70f None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.073538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0e12f-c9f2-403f-b5bd-3ab39768e70f None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.073809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0e12f-c9f2-403f-b5bd-3ab39768e70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.094691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50783e-fe97-4a60-a85b-e214c49d34e3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.101786 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-f7b0e12f-c9f2-403f-b5bd-3ab39768e70f None None] MFA Rules not processed for user `5fb488cb6e2d443db89acfb629501d82`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:21.103028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f7b0e12f-c9f2-403f-b5bd-3ab39768e70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:21.151224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c6f1d-85ab-469e-9a9d-c09a4c35efc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.151954 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c6f1d-85ab-469e-9a9d-c09a4c35efc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.152915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c6f1d-85ab-469e-9a9d-c09a4c35efc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.153709 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c6f1d-85ab-469e-9a9d-c09a4c35efc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.154696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c6f1d-85ab-469e-9a9d-c09a4c35efc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.156307 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c6f1d-85ab-469e-9a9d-c09a4c35efc7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.173336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 783/1580] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:21.174219 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0e12f-c9f2-403f-b5bd-3ab39768e70f None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.183974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0e12f-c9f2-403f-b5bd-3ab39768e70f None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.188499 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b5f59-ad67-4320-ae5a-b0f91454b170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.191080 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b5f59-ad67-4320-ae5a-b0f91454b170 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.191080 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b5f59-ad67-4320-ae5a-b0f91454b170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.191080 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b5f59-ad67-4320-ae5a-b0f91454b170 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.191080 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b5f59-ad67-4320-ae5a-b0f91454b170 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.192147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b5f59-ad67-4320-ae5a-b0f91454b170 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.241683 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 798/1581] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:13:21 2022] POST /identity/v3/users => generated 294 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:21.243005 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e331b5-7767-4529-8c36-b1ea75264025 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.243241 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e331b5-7767-4529-8c36-b1ea75264025 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.243511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e331b5-7767-4529-8c36-b1ea75264025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.271684 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-76e331b5-7767-4529-8c36-b1ea75264025 None None] MFA Rules not processed for user `3907dcef06814ce7a1434d9c451c1980`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:21.272904 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-76e331b5-7767-4529-8c36-b1ea75264025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:21.276959 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:21.310598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:21.317904 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 784/1582] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:13:21 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:21.319115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b5f59-ad67-4320-ae5a-b0f91454b170 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.329512 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b5f59-ad67-4320-ae5a-b0f91454b170 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.334176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec63f68-3fc6-4544-9fbe-4a37276c6028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.334758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec63f68-3fc6-4544-9fbe-4a37276c6028 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.335177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec63f68-3fc6-4544-9fbe-4a37276c6028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.335448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec63f68-3fc6-4544-9fbe-4a37276c6028 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.336065 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec63f68-3fc6-4544-9fbe-4a37276c6028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.337598 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 799/1583] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:21.338050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec63f68-3fc6-4544-9fbe-4a37276c6028 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.338373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e331b5-7767-4529-8c36-b1ea75264025 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.354132 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e331b5-7767-4529-8c36-b1ea75264025 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.359114 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ce037-1f43-4631-ad7c-e5ec4419b046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.359715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ce037-1f43-4631-ad7c-e5ec4419b046 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.359996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ce037-1f43-4631-ad7c-e5ec4419b046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.360287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ce037-1f43-4631-ad7c-e5ec4419b046 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.360903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ce037-1f43-4631-ad7c-e5ec4419b046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.363048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ce037-1f43-4631-ad7c-e5ec4419b046 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.447026 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 800/1584] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:13:21 2022] POST /identity/v3/users => generated 283 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:21.447965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 785/1585] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:13:21 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:21.448783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec63f68-3fc6-4544-9fbe-4a37276c6028 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.453985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ce037-1f43-4631-ad7c-e5ec4419b046 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.459348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec63f68-3fc6-4544-9fbe-4a37276c6028 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.463473 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ce037-1f43-4631-ad7c-e5ec4419b046 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.463991 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d796bc2-7512-4757-9400-74115129eb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.464577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d796bc2-7512-4757-9400-74115129eb89 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.465487 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d796bc2-7512-4757-9400-74115129eb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.465767 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d796bc2-7512-4757-9400-74115129eb89 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.466395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d796bc2-7512-4757-9400-74115129eb89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.468492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d796bc2-7512-4757-9400-74115129eb89 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.469124 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73811707-1468-404b-a6b1-581ea3ba1b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.469708 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73811707-1468-404b-a6b1-581ea3ba1b64 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.469984 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73811707-1468-404b-a6b1-581ea3ba1b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.470254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73811707-1468-404b-a6b1-581ea3ba1b64 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.470875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73811707-1468-404b-a6b1-581ea3ba1b64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.472837 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73811707-1468-404b-a6b1-581ea3ba1b64 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.584384 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 786/1586] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:13:21 2022] POST /identity/v3/users => generated 297 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:21.585354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d796bc2-7512-4757-9400-74115129eb89 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.594908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d796bc2-7512-4757-9400-74115129eb89 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.599487 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae890cd-a504-4705-90f4-501854ac28bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.600098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae890cd-a504-4705-90f4-501854ac28bb admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.600371 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae890cd-a504-4705-90f4-501854ac28bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.600637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae890cd-a504-4705-90f4-501854ac28bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.601240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae890cd-a504-4705-90f4-501854ac28bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.603161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae890cd-a504-4705-90f4-501854ac28bb admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.613358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 801/1587] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:13:21 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:21.619112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73811707-1468-404b-a6b1-581ea3ba1b64 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.624786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73811707-1468-404b-a6b1-581ea3ba1b64 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.629717 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda007f4-31e7-4cde-86c7-36425c8cf5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.630306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda007f4-31e7-4cde-86c7-36425c8cf5e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.630582 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda007f4-31e7-4cde-86c7-36425c8cf5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.630852 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda007f4-31e7-4cde-86c7-36425c8cf5e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.631600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda007f4-31e7-4cde-86c7-36425c8cf5e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.633191 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda007f4-31e7-4cde-86c7-36425c8cf5e7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.644415 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 802/1588] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:21.647700 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda007f4-31e7-4cde-86c7-36425c8cf5e7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.654300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda007f4-31e7-4cde-86c7-36425c8cf5e7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.658857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cabdef-cc05-4a8e-8ced-2c74b3c43bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.659443 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cabdef-cc05-4a8e-8ced-2c74b3c43bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.659722 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cabdef-cc05-4a8e-8ced-2c74b3c43bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.659992 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cabdef-cc05-4a8e-8ced-2c74b3c43bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.660596 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cabdef-cc05-4a8e-8ced-2c74b3c43bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.662116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cabdef-cc05-4a8e-8ced-2c74b3c43bc4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.672532 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 803/1589] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:21.674062 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0207b-9d2c-4c99-a646-5e920f1bfeb1 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.674349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0207b-9d2c-4c99-a646-5e920f1bfeb1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.674618 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0207b-9d2c-4c99-a646-5e920f1bfeb1 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.676014 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 804/1590] 10.222.0.30 () {54 vars in 961 bytes} [Thu Oct 6 08:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:13:21.676963 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0207b-9d2c-4c99-a646-5e920f1bfeb1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.692383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0207b-9d2c-4c99-a646-5e920f1bfeb1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.696895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e954c9-0d74-44cc-9494-dadcac10c641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.697526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e954c9-0d74-44cc-9494-dadcac10c641 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.697751 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e954c9-0d74-44cc-9494-dadcac10c641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.698017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e954c9-0d74-44cc-9494-dadcac10c641 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.698644 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e954c9-0d74-44cc-9494-dadcac10c641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.700956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e954c9-0d74-44cc-9494-dadcac10c641 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.711083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 805/1591] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:21.712023 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e954c9-0d74-44cc-9494-dadcac10c641 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.716714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 787/1592] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:13:21 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:21.716714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae890cd-a504-4705-90f4-501854ac28bb admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.722265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e954c9-0d74-44cc-9494-dadcac10c641 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.776492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae890cd-a504-4705-90f4-501854ac28bb admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.779988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc623b44-c5cc-493d-b69e-9bae2018efa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.780603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc623b44-c5cc-493d-b69e-9bae2018efa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.780878 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc623b44-c5cc-493d-b69e-9bae2018efa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.781146 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc623b44-c5cc-493d-b69e-9bae2018efa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.781354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c305c7-e513-46c9-950a-4eebcd3f984c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.781790 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc623b44-c5cc-493d-b69e-9bae2018efa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.781968 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c305c7-e513-46c9-950a-4eebcd3f984c admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.782229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c305c7-e513-46c9-950a-4eebcd3f984c admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.782499 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c305c7-e513-46c9-950a-4eebcd3f984c admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.783175 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c305c7-e513-46c9-950a-4eebcd3f984c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.783451 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc623b44-c5cc-493d-b69e-9bae2018efa2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.784760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c305c7-e513-46c9-950a-4eebcd3f984c admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.793511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 806/1593] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:21.794321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc623b44-c5cc-493d-b69e-9bae2018efa2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.797615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 788/1594] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:21.797615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c305c7-e513-46c9-950a-4eebcd3f984c admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.805483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc623b44-c5cc-493d-b69e-9bae2018efa2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.806653 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c305c7-e513-46c9-950a-4eebcd3f984c admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.810724 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5b4fa-0a11-4a81-a2b7-239833ea6381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.811345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5b4fa-0a11-4a81-a2b7-239833ea6381 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.811637 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5b4fa-0a11-4a81-a2b7-239833ea6381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.811886 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5b4fa-0a11-4a81-a2b7-239833ea6381 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.812199 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27ba3a-e2c8-4206-9f71-2f1021a57646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.812551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5b4fa-0a11-4a81-a2b7-239833ea6381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.812835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27ba3a-e2c8-4206-9f71-2f1021a57646 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.813112 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27ba3a-e2c8-4206-9f71-2f1021a57646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.813382 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27ba3a-e2c8-4206-9f71-2f1021a57646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.814009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27ba3a-e2c8-4206-9f71-2f1021a57646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.814143 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5b4fa-0a11-4a81-a2b7-239833ea6381 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.815713 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27ba3a-e2c8-4206-9f71-2f1021a57646 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.825343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 807/1595] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:21.826057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5b4fa-0a11-4a81-a2b7-239833ea6381 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.830304 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 789/1596] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:21.831815 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77012713-9d78-40a2-80be-d98d577e6a07 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.832705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77012713-9d78-40a2-80be-d98d577e6a07 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.832988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77012713-9d78-40a2-80be-d98d577e6a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.835745 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5b4fa-0a11-4a81-a2b7-239833ea6381 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.840439 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5815c-a344-4763-b644-587489c48f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.841024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5815c-a344-4763-b644-587489c48f64 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.841304 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5815c-a344-4763-b644-587489c48f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.841572 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5815c-a344-4763-b644-587489c48f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.842197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5815c-a344-4763-b644-587489c48f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.843807 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5815c-a344-4763-b644-587489c48f64 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.853602 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 808/1597] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:21.854397 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5815c-a344-4763-b644-587489c48f64 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.863729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5815c-a344-4763-b644-587489c48f64 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.868325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55577598-3bce-4674-a618-8e7d37de1357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.868900 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55577598-3bce-4674-a618-8e7d37de1357 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.869177 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55577598-3bce-4674-a618-8e7d37de1357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.869447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55577598-3bce-4674-a618-8e7d37de1357 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.870072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55577598-3bce-4674-a618-8e7d37de1357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.871738 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-77012713-9d78-40a2-80be-d98d577e6a07 None None] MFA Rules not processed for user `209795285384486183cda455fe316bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:21.872475 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55577598-3bce-4674-a618-8e7d37de1357 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.873048 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-77012713-9d78-40a2-80be-d98d577e6a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:21.877721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:21.882866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 809/1598] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:21.890131 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55577598-3bce-4674-a618-8e7d37de1357 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.896378 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55577598-3bce-4674-a618-8e7d37de1357 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.901336 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8ee47-6ec9-45cd-b31a-f7194112bbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.901949 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8ee47-6ec9-45cd-b31a-f7194112bbe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.902253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8ee47-6ec9-45cd-b31a-f7194112bbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.902663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8ee47-6ec9-45cd-b31a-f7194112bbe9 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.903335 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8ee47-6ec9-45cd-b31a-f7194112bbe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.905407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8ee47-6ec9-45cd-b31a-f7194112bbe9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.944128 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 790/1599] 10.222.0.30 () {58 vars in 1058 bytes} [Thu Oct 6 08:13:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:21.944970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77012713-9d78-40a2-80be-d98d577e6a07 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.951402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:21.955415 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77012713-9d78-40a2-80be-d98d577e6a07 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.960119 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8616889-b96e-45d3-b57b-fe198126030a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.960723 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8616889-b96e-45d3-b57b-fe198126030a admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.961003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8616889-b96e-45d3-b57b-fe198126030a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.961278 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8616889-b96e-45d3-b57b-fe198126030a admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.961920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8616889-b96e-45d3-b57b-fe198126030a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.963561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8616889-b96e-45d3-b57b-fe198126030a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:21.973625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 791/1600] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:21.974418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8616889-b96e-45d3-b57b-fe198126030a admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:21.984281 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8616889-b96e-45d3-b57b-fe198126030a admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:21.988863 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b21bb-f636-4724-b4d6-ca09ad61f80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:21.993982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b21bb-f636-4724-b4d6-ca09ad61f80d admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:21.993982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b21bb-f636-4724-b4d6-ca09ad61f80d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:21.993982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b21bb-f636-4724-b4d6-ca09ad61f80d admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:21.993982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b21bb-f636-4724-b4d6-ca09ad61f80d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:21.993982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b21bb-f636-4724-b4d6-ca09ad61f80d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:22.002509 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 792/1601] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:22.003309 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b21bb-f636-4724-b4d6-ca09ad61f80d admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:22.012902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b21bb-f636-4724-b4d6-ca09ad61f80d admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:22.017441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6730c2-dde2-4b72-80bf-ceb454f16e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:22.017990 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6730c2-dde2-4b72-80bf-ceb454f16e59 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.018267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6730c2-dde2-4b72-80bf-ceb454f16e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.018536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6730c2-dde2-4b72-80bf-ceb454f16e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.019171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6730c2-dde2-4b72-80bf-ceb454f16e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:22.020712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6730c2-dde2-4b72-80bf-ceb454f16e59 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:22.030931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 793/1602] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:22.031733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6730c2-dde2-4b72-80bf-ceb454f16e59 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:22.040840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6730c2-dde2-4b72-80bf-ceb454f16e59 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:22.045825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14bd6a-3b6d-4470-9922-0394734a8906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:22.046422 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14bd6a-3b6d-4470-9922-0394734a8906 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.046692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14bd6a-3b6d-4470-9922-0394734a8906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.046971 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14bd6a-3b6d-4470-9922-0394734a8906 admin admin] PATH_INFO: `/v3/projects/95f59687b3b24c7f9095bcb5fb909a67/users/36b26040882a4e85b82b40a3c17d33bb/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.061934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14bd6a-3b6d-4470-9922-0394734a8906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f59687b3b24c7f9095bcb5fb909a67, user_id=36b26040882a4e85b82b40a3c17d33bb, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:22.064916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14bd6a-3b6d-4470-9922-0394734a8906 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:22.068865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 810/1603] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:13:21 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:22.068865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8ee47-6ec9-45cd-b31a-f7194112bbe9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:22.076872 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8ee47-6ec9-45cd-b31a-f7194112bbe9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:22.081495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ccbef-8e55-4c77-b624-083835cea7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:22.082129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ccbef-8e55-4c77-b624-083835cea7ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.082403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ccbef-8e55-4c77-b624-083835cea7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.082674 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ccbef-8e55-4c77-b624-083835cea7ab admin admin] PATH_INFO: `/v3/projects/73f960dc36054e9ba7003628a82b7039/users/cb73bbd331754280a2a0cd64290ec3c8/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.097740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ccbef-8e55-4c77-b624-083835cea7ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=73f960dc36054e9ba7003628a82b7039, user_id=cb73bbd331754280a2a0cd64290ec3c8, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:22.100474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ccbef-8e55-4c77-b624-083835cea7ab admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:22.150165 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 794/1604] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:13:22 2022] PUT /identity/v3/projects/95f59687b3b24c7f9095bcb5fb909a67/users/36b26040882a4e85b82b40a3c17d33bb/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:22.150911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14bd6a-3b6d-4470-9922-0394734a8906 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:22.185897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 811/1605] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:13:22 2022] PUT /identity/v3/projects/73f960dc36054e9ba7003628a82b7039/users/cb73bbd331754280a2a0cd64290ec3c8/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:22.186737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ccbef-8e55-4c77-b624-083835cea7ab admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:22.214634 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14bd6a-3b6d-4470-9922-0394734a8906 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:22.259757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ccbef-8e55-4c77-b624-083835cea7ab admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:22.265311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c945be-c8ab-476e-90a9-5f8e432238e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:22.265907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c945be-c8ab-476e-90a9-5f8e432238e6 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.266178 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c945be-c8ab-476e-90a9-5f8e432238e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.266445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c945be-c8ab-476e-90a9-5f8e432238e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.272714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf2616-d56a-4a7d-a498-1a966b392297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:22.273367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf2616-d56a-4a7d-a498-1a966b392297 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.273644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf2616-d56a-4a7d-a498-1a966b392297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.273917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf2616-d56a-4a7d-a498-1a966b392297 admin admin] PATH_INFO: `/v3/projects/74ade644bb9a4efe8ebb365a3d076522/users/9180d83ed50f4236b25c5cb73f478686/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.288667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf2616-d56a-4a7d-a498-1a966b392297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74ade644bb9a4efe8ebb365a3d076522, user_id=9180d83ed50f4236b25c5cb73f478686, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:22.291372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf2616-d56a-4a7d-a498-1a966b392297 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:22.322080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c945be-c8ab-476e-90a9-5f8e432238e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:22.323853 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c945be-c8ab-476e-90a9-5f8e432238e6 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:22.332747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 812/1606] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:22.338453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c945be-c8ab-476e-90a9-5f8e432238e6 service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:22.343738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c945be-c8ab-476e-90a9-5f8e432238e6 service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:22.348611 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87040004-aa75-4469-a9a9-a9a51c35ae49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:22.349244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87040004-aa75-4469-a9a9-a9a51c35ae49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.349524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87040004-aa75-4469-a9a9-a9a51c35ae49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.349824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87040004-aa75-4469-a9a9-a9a51c35ae49 admin admin] PATH_INFO: `/v3/projects/37c1a79616054d3c988fac816f4acc8b/users/9427fe8723fb4b298b42106a862a798f/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.362522 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 795/1607] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:13:22 2022] PUT /identity/v3/projects/74ade644bb9a4efe8ebb365a3d076522/users/9180d83ed50f4236b25c5cb73f478686/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:22.363897 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf2616-d56a-4a7d-a498-1a966b392297 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:22.375624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87040004-aa75-4469-a9a9-a9a51c35ae49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37c1a79616054d3c988fac816f4acc8b, user_id=9427fe8723fb4b298b42106a862a798f, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:22.378773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87040004-aa75-4469-a9a9-a9a51c35ae49 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:22.411813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:22.432597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf2616-d56a-4a7d-a498-1a966b392297 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:22.437271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f2f6c-0124-4cf8-a0ec-d6ea7fef1631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:22.437917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f2f6c-0124-4cf8-a0ec-d6ea7fef1631 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.438190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f2f6c-0124-4cf8-a0ec-d6ea7fef1631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.438454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f2f6c-0124-4cf8-a0ec-d6ea7fef1631 admin admin] PATH_INFO: `/v3/projects/d5f023bd0dcf41558003e085c1dddf30/users/85bdfc4b6ad443d68ebc0d6932ccad66/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.448207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 813/1608] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:13:22 2022] PUT /identity/v3/projects/37c1a79616054d3c988fac816f4acc8b/users/9427fe8723fb4b298b42106a862a798f/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:22.449500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87040004-aa75-4469-a9a9-a9a51c35ae49 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:22.453957 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f2f6c-0124-4cf8-a0ec-d6ea7fef1631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5f023bd0dcf41558003e085c1dddf30, user_id=85bdfc4b6ad443d68ebc0d6932ccad66, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:22.456686 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f2f6c-0124-4cf8-a0ec-d6ea7fef1631 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:22.492148 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:22.521208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87040004-aa75-4469-a9a9-a9a51c35ae49 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:22.525833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 796/1609] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:13:22 2022] PUT /identity/v3/projects/d5f023bd0dcf41558003e085c1dddf30/users/85bdfc4b6ad443d68ebc0d6932ccad66/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:22.525833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f2f6c-0124-4cf8-a0ec-d6ea7fef1631 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:22.584544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a7998-a090-4053-9219-69e09789e29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:22.585208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a7998-a090-4053-9219-69e09789e29b admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.585485 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a7998-a090-4053-9219-69e09789e29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.585756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a7998-a090-4053-9219-69e09789e29b admin admin] PATH_INFO: `/v3/projects/d6a6c776cba64be9a1710815c0b0f952/users/9932c7fb57394d709a27023f3ae2677e/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.595376 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f2f6c-0124-4cf8-a0ec-d6ea7fef1631 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:22.600160 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74d9bc-f4e5-4a08-981f-ba7c408fcc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:22.600595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a7998-a090-4053-9219-69e09789e29b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6a6c776cba64be9a1710815c0b0f952, user_id=9932c7fb57394d709a27023f3ae2677e, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:22.600772 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74d9bc-f4e5-4a08-981f-ba7c408fcc51 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.601036 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74d9bc-f4e5-4a08-981f-ba7c408fcc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.601305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74d9bc-f4e5-4a08-981f-ba7c408fcc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.601946 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74d9bc-f4e5-4a08-981f-ba7c408fcc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:22.603328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a7998-a090-4053-9219-69e09789e29b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:22.603699 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74d9bc-f4e5-4a08-981f-ba7c408fcc51 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:22.614818 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 797/1610] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:22.619034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633202c-6407-418c-a4a5-fa7d2e92d368 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.619034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633202c-6407-418c-a4a5-fa7d2e92d368 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.619034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633202c-6407-418c-a4a5-fa7d2e92d368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.635682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-3633202c-6407-418c-a4a5-fa7d2e92d368 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:22.637541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3633202c-6407-418c-a4a5-fa7d2e92d368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:22.645176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 798/1611] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:22.646345 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef61bf2-0913-44bf-894e-b60cb419e4ff None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.646620 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef61bf2-0913-44bf-894e-b60cb419e4ff None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.646888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef61bf2-0913-44bf-894e-b60cb419e4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.663876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 814/1612] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:13:22 2022] PUT /identity/v3/projects/d6a6c776cba64be9a1710815c0b0f952/users/9932c7fb57394d709a27023f3ae2677e/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:22.665133 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbb5c2-eaaa-4bce-a182-a5ba141c6fe3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.665416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbb5c2-eaaa-4bce-a182-a5ba141c6fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.665805 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbb5c2-eaaa-4bce-a182-a5ba141c6fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.666395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-0ef61bf2-0913-44bf-894e-b60cb419e4ff None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:22.667690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0ef61bf2-0913-44bf-894e-b60cb419e4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:22.694351 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5bcbb5c2-eaaa-4bce-a182-a5ba141c6fe3 None None] MFA Rules not processed for user `9180d83ed50f4236b25c5cb73f478686`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:22.695601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5bcbb5c2-eaaa-4bce-a182-a5ba141c6fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:22.729289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 799/1613] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:22.731293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4c04f-d0ef-4a5d-aa50-571f9f6ab68a None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.731293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4c04f-d0ef-4a5d-aa50-571f9f6ab68a None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.731293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4c04f-d0ef-4a5d-aa50-571f9f6ab68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.750736 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-1bf4c04f-d0ef-4a5d-aa50-571f9f6ab68a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:22.751986 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1bf4c04f-d0ef-4a5d-aa50-571f9f6ab68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:22.759729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 800/1614] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:22.760970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4552b-6900-4f7a-aae2-15837bf6f099 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.761278 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4552b-6900-4f7a-aae2-15837bf6f099 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.762054 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4552b-6900-4f7a-aae2-15837bf6f099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.765682 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 815/1615] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:22.766967 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b43c1-00b8-4827-bd0a-718d3db3443a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.767253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b43c1-00b8-4827-bd0a-718d3db3443a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.767519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b43c1-00b8-4827-bd0a-718d3db3443a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.769514 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 816/1616] 10.222.0.30 () {54 vars in 912 bytes} [Thu Oct 6 08:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:13:22.772080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b43c1-00b8-4827-bd0a-718d3db3443a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:22.786412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-70a4552b-6900-4f7a-aae2-15837bf6f099 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:22.786602 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b43c1-00b8-4827-bd0a-718d3db3443a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:22.787620 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-70a4552b-6900-4f7a-aae2-15837bf6f099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:22.791093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c006b4-7f7d-479e-9501-7ca92be9c01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:22.791735 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c006b4-7f7d-479e-9501-7ca92be9c01c admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.792009 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c006b4-7f7d-479e-9501-7ca92be9c01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.792279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c006b4-7f7d-479e-9501-7ca92be9c01c admin admin] PATH_INFO: `/v3/projects/d56a5f5ad43842ff8e560562be508578/users/f11014add5d64419b506ad4a1b3647c6/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.795428 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 801/1617] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:22.796856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4552b-6900-4f7a-aae2-15837bf6f099 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:22.811906 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c006b4-7f7d-479e-9501-7ca92be9c01c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56a5f5ad43842ff8e560562be508578, user_id=f11014add5d64419b506ad4a1b3647c6, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:22.815248 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c006b4-7f7d-479e-9501-7ca92be9c01c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:22.843180 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:22.856432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4552b-6900-4f7a-aae2-15837bf6f099 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:22.860046 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee578b2-0964-4e77-8470-f54d8ba87f8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:22.860641 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee578b2-0964-4e77-8470-f54d8ba87f8e service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.860913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee578b2-0964-4e77-8470-f54d8ba87f8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.861178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee578b2-0964-4e77-8470-f54d8ba87f8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.869941 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee578b2-0964-4e77-8470-f54d8ba87f8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:22.871696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee578b2-0964-4e77-8470-f54d8ba87f8e service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:22.881048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 817/1618] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:13:22 2022] PUT /identity/v3/projects/d56a5f5ad43842ff8e560562be508578/users/f11014add5d64419b506ad4a1b3647c6/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:22.882240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 802/1619] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:22.882459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d640f-6917-468c-a8ed-6b6fa827d75b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.882744 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d640f-6917-468c-a8ed-6b6fa827d75b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.883036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d640f-6917-468c-a8ed-6b6fa827d75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.883140 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee578b2-0964-4e77-8470-f54d8ba87f8e service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:22.892292 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee578b2-0964-4e77-8470-f54d8ba87f8e service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:22.903751 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9d7d640f-6917-468c-a8ed-6b6fa827d75b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:22.905048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9d7d640f-6917-468c-a8ed-6b6fa827d75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:22.925212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:22.941622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d90e69-04e2-4780-acd5-2258d7bee4de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:22.942467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d90e69-04e2-4780-acd5-2258d7bee4de service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.942600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d90e69-04e2-4780-acd5-2258d7bee4de service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.942784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d90e69-04e2-4780-acd5-2258d7bee4de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:22.963721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 818/1620] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:22.964560 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d640f-6917-468c-a8ed-6b6fa827d75b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:22.975093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d640f-6917-468c-a8ed-6b6fa827d75b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:22.989252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61379f52-63e3-4c54-b82c-821bafdf8635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:22.990129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61379f52-63e3-4c54-b82c-821bafdf8635 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:22.991013 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61379f52-63e3-4c54-b82c-821bafdf8635 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:22.996917 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61379f52-63e3-4c54-b82c-821bafdf8635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:23.005728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61379f52-63e3-4c54-b82c-821bafdf8635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:23.007643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61379f52-63e3-4c54-b82c-821bafdf8635 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:23.012438 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d90e69-04e2-4780-acd5-2258d7bee4de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:23.013974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d90e69-04e2-4780-acd5-2258d7bee4de service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:23.020120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 819/1621] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:23.021929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ada2d-afb9-4258-b8dd-1ba05f16f842 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:23.022426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ada2d-afb9-4258-b8dd-1ba05f16f842 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:23.022850 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ada2d-afb9-4258-b8dd-1ba05f16f842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:23.024378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 803/1622] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:23.025233 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d90e69-04e2-4780-acd5-2258d7bee4de service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:23.048621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-845ada2d-afb9-4258-b8dd-1ba05f16f842 None None] MFA Rules not processed for user `209795285384486183cda455fe316bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:23.050252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-845ada2d-afb9-4258-b8dd-1ba05f16f842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:23.090257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d90e69-04e2-4780-acd5-2258d7bee4de service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:23.094194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a021bb-06c9-45a3-a881-44931fddbefe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:23.094802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a021bb-06c9-45a3-a881-44931fddbefe service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:23.095109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a021bb-06c9-45a3-a881-44931fddbefe service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:23.095386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a021bb-06c9-45a3-a881-44931fddbefe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:23.137606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 820/1623] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Oct 6 08:13:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:23.139059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ada2d-afb9-4258-b8dd-1ba05f16f842 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:23.152755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:23.157986 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ada2d-afb9-4258-b8dd-1ba05f16f842 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:23.162753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25a8f6-36c3-4f36-982c-2d28e2a2b055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:23.163397 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25a8f6-36c3-4f36-982c-2d28e2a2b055 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:23.163780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25a8f6-36c3-4f36-982c-2d28e2a2b055 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:23.164052 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25a8f6-36c3-4f36-982c-2d28e2a2b055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:23.169077 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a021bb-06c9-45a3-a881-44931fddbefe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:23.171139 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a021bb-06c9-45a3-a881-44931fddbefe service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:23.175873 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25a8f6-36c3-4f36-982c-2d28e2a2b055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:23.177334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25a8f6-36c3-4f36-982c-2d28e2a2b055 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:23.190845 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 821/1624] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:23.192051 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25a8f6-36c3-4f36-982c-2d28e2a2b055 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:23.209032 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 804/1625] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:23.209032 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a021bb-06c9-45a3-a881-44931fddbefe service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:23.209433 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25a8f6-36c3-4f36-982c-2d28e2a2b055 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:23.210794 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac163dba-576b-4f80-b0c6-485f790fd1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:23.211900 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac163dba-576b-4f80-b0c6-485f790fd1c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:23.212181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac163dba-576b-4f80-b0c6-485f790fd1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:23.212932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac163dba-576b-4f80-b0c6-485f790fd1c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:23.214460 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac163dba-576b-4f80-b0c6-485f790fd1c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:23.216281 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac163dba-576b-4f80-b0c6-485f790fd1c3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:23.227376 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a021bb-06c9-45a3-a881-44931fddbefe service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:23.247889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 822/1626] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:23.247889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac163dba-576b-4f80-b0c6-485f790fd1c3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:23.247889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac163dba-576b-4f80-b0c6-485f790fd1c3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:23.249360 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ccd01-c2e7-4404-80cf-14bcf125a573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:23.250592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaace18-f38a-4bc1-9c85-23f539ab9067 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:23.250800 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ccd01-c2e7-4404-80cf-14bcf125a573 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:23.251288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaace18-f38a-4bc1-9c85-23f539ab9067 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:23.251558 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaace18-f38a-4bc1-9c85-23f539ab9067 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:23.251829 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaace18-f38a-4bc1-9c85-23f539ab9067 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:23.253203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ccd01-c2e7-4404-80cf-14bcf125a573 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:23.254419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ccd01-c2e7-4404-80cf-14bcf125a573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:23.267183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaace18-f38a-4bc1-9c85-23f539ab9067 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:23.268850 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaace18-f38a-4bc1-9c85-23f539ab9067 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:23.279962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ccd01-c2e7-4404-80cf-14bcf125a573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:23.281764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ccd01-c2e7-4404-80cf-14bcf125a573 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:23.289371 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 823/1627] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:23.290349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaace18-f38a-4bc1-9c85-23f539ab9067 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:23.305188 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaace18-f38a-4bc1-9c85-23f539ab9067 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:23.312455 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb8330-073e-489a-9755-c87920fc9571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:23.313483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb8330-073e-489a-9755-c87920fc9571 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:23.313871 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb8330-073e-489a-9755-c87920fc9571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:23.314239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb8330-073e-489a-9755-c87920fc9571 admin admin] PATH_INFO: `/v3/projects/d56a5f5ad43842ff8e560562be508578/users/f11014add5d64419b506ad4a1b3647c6/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:23.316213 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 805/1628] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:23.316920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb8330-073e-489a-9755-c87920fc9571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56a5f5ad43842ff8e560562be508578, user_id=f11014add5d64419b506ad4a1b3647c6, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:23.320079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb8330-073e-489a-9755-c87920fc9571 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:23.337958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 824/1629] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:13:23 2022] PUT /identity/v3/projects/d56a5f5ad43842ff8e560562be508578/users/f11014add5d64419b506ad4a1b3647c6/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:23.353046 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4ec15-8a31-478c-aaba-199eca7919fc None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:23.353399 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4ec15-8a31-478c-aaba-199eca7919fc None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:23.353642 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4ec15-8a31-478c-aaba-199eca7919fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:23.407810 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-5be4ec15-8a31-478c-aaba-199eca7919fc None None] MFA Rules not processed for user `f11014add5d64419b506ad4a1b3647c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:23.410397 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5be4ec15-8a31-478c-aaba-199eca7919fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:23.577401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 806/1630] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:24.257026 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb8330-073e-489a-9755-c87920fc9571 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:24.317403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb8330-073e-489a-9755-c87920fc9571 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:24.321199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaedb0f0-7b39-4e8b-90a2-c8db1239527f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:24.321865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaedb0f0-7b39-4e8b-90a2-c8db1239527f service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:24.322145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaedb0f0-7b39-4e8b-90a2-c8db1239527f service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:24.322413 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaedb0f0-7b39-4e8b-90a2-c8db1239527f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:24.339104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaedb0f0-7b39-4e8b-90a2-c8db1239527f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:24.350274 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaedb0f0-7b39-4e8b-90a2-c8db1239527f service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:24.355851 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:24.364515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 825/1631] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:24.998638 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc569a0-de51-4d9a-9d09-45b5dfc5b0d0 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:24.998638 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc569a0-de51-4d9a-9d09-45b5dfc5b0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:24.998638 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc569a0-de51-4d9a-9d09-45b5dfc5b0d0 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:24.998638 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 807/1632] 10.222.0.30 () {54 vars in 961 bytes} [Thu Oct 6 08:13:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:13:25.005416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fd69a-9b51-436a-943d-095b577c0aba None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:25.005416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fd69a-9b51-436a-943d-095b577c0aba None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:25.005416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fd69a-9b51-436a-943d-095b577c0aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:25.005416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ec9fd69a-9b51-436a-943d-095b577c0aba None None] MFA Rules not processed for user `209795285384486183cda455fe316bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:25.005416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ec9fd69a-9b51-436a-943d-095b577c0aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:25.009063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc569a0-de51-4d9a-9d09-45b5dfc5b0d0 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:25.009063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc569a0-de51-4d9a-9d09-45b5dfc5b0d0 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:25.009063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877ca1d-0765-407c-b8a8-120a562eb85f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:25.009063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877ca1d-0765-407c-b8a8-120a562eb85f service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:25.010397 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877ca1d-0765-407c-b8a8-120a562eb85f service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:25.010397 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877ca1d-0765-407c-b8a8-120a562eb85f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:25.029864 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 826/1633] 10.222.0.30 () {58 vars in 1058 bytes} [Thu Oct 6 08:13:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:25.034621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e3b2e-8290-4b31-a2f5-7f99bd4edd8a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:25.034904 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e3b2e-8290-4b31-a2f5-7f99bd4edd8a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:25.035473 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e3b2e-8290-4b31-a2f5-7f99bd4edd8a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:25.037926 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 827/1634] 10.222.0.30 () {54 vars in 912 bytes} [Thu Oct 6 08:13:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:13:25.044668 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcfdf4-46e1-4c36-a016-386fb67df3a7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:25.044950 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcfdf4-46e1-4c36-a016-386fb67df3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:25.047846 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcfdf4-46e1-4c36-a016-386fb67df3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:25.075747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877ca1d-0765-407c-b8a8-120a562eb85f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:25.078833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877ca1d-0765-407c-b8a8-120a562eb85f service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:25.091742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 808/1635] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:25.092759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-99dcfdf4-46e1-4c36-a016-386fb67df3a7 None None] MFA Rules not processed for user `209795285384486183cda455fe316bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:25.094588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-99dcfdf4-46e1-4c36-a016-386fb67df3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:25.107175 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 828/1636] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Oct 6 08:13:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:13:25.125103 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877ca1d-0765-407c-b8a8-120a562eb85f service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:25.134212 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877ca1d-0765-407c-b8a8-120a562eb85f service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:25.143398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd217a8-9115-4dea-8c48-da2fbdbced16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:25.143398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd217a8-9115-4dea-8c48-da2fbdbced16 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:25.143398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd217a8-9115-4dea-8c48-da2fbdbced16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:25.143398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd217a8-9115-4dea-8c48-da2fbdbced16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:25.157248 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd217a8-9115-4dea-8c48-da2fbdbced16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:25.158694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd217a8-9115-4dea-8c48-da2fbdbced16 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:25.176505 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 809/1637] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:31.280243 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcfdf4-46e1-4c36-a016-386fb67df3a7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:31.504083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcfdf4-46e1-4c36-a016-386fb67df3a7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:31.525866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c415355-dc17-4871-86ef-632c9b7f6423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:31.526927 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c415355-dc17-4871-86ef-632c9b7f6423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:31.527417 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c415355-dc17-4871-86ef-632c9b7f6423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:31.527869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c415355-dc17-4871-86ef-632c9b7f6423 admin admin] PATH_INFO: `/v3/users/3907dcef06814ce7a1434d9c451c1980` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:31.529740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c415355-dc17-4871-86ef-632c9b7f6423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3907dcef06814ce7a1434d9c451c1980)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:31.531756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c415355-dc17-4871-86ef-632c9b7f6423 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:31.847330 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5c415355-dc17-4871-86ef-632c9b7f6423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3907dcef06814ce7a1434d9c451c1980'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:31.890641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5c415355-dc17-4871-86ef-632c9b7f6423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3907dcef06814ce7a1434d9c451c1980'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:31.891238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5c415355-dc17-4871-86ef-632c9b7f6423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3907dcef06814ce7a1434d9c451c1980'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:32.020026 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5c415355-dc17-4871-86ef-632c9b7f6423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3907dcef06814ce7a1434d9c451c1980'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:32.021880 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5c415355-dc17-4871-86ef-632c9b7f6423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3907dcef06814ce7a1434d9c451c1980'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:32.071467 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 829/1638] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:13:31 2022] DELETE /identity/v3/users/3907dcef06814ce7a1434d9c451c1980 => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:32.077369 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce7979-cc56-4cf7-8368-60041c862fd3 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:32.078520 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce7979-cc56-4cf7-8368-60041c862fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:32.078963 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce7979-cc56-4cf7-8368-60041c862fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:32.125636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-9dce7979-cc56-4cf7-8368-60041c862fd3 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:32.127346 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9dce7979-cc56-4cf7-8368-60041c862fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:32.240173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:32.309755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 810/1639] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:32.527716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c415355-dc17-4871-86ef-632c9b7f6423 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:32.530170 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5c415355-dc17-4871-86ef-632c9b7f6423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:32.539191 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:32.725539 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c415355-dc17-4871-86ef-632c9b7f6423 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:32.729711 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764501d-fa36-4868-84af-db446a24809b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:32.730342 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764501d-fa36-4868-84af-db446a24809b service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:32.730692 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764501d-fa36-4868-84af-db446a24809b service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:32.731043 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764501d-fa36-4868-84af-db446a24809b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:32.757365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764501d-fa36-4868-84af-db446a24809b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:32.758931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764501d-fa36-4868-84af-db446a24809b service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:32.787572 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 830/1640] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:36.237100 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce7979-cc56-4cf7-8368-60041c862fd3 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:36.240073 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9dce7979-cc56-4cf7-8368-60041c862fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:36.267450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce7979-cc56-4cf7-8368-60041c862fd3 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:36.284649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c054ff-3b69-4e1f-aa67-795d3a1772f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:36.284979 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c054ff-3b69-4e1f-aa67-795d3a1772f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:36.285056 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c054ff-3b69-4e1f-aa67-795d3a1772f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:36.285424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c054ff-3b69-4e1f-aa67-795d3a1772f3 admin admin] PATH_INFO: `/v3/projects/546cd66807054a5a8d7a8137e90a4f36` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:36.287618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c054ff-3b69-4e1f-aa67-795d3a1772f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=546cd66807054a5a8d7a8137e90a4f36)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:36.292041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c054ff-3b69-4e1f-aa67-795d3a1772f3 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:36.446886 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-50c054ff-3b69-4e1f-aa67-795d3a1772f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '546cd66807054a5a8d7a8137e90a4f36', 'request_id': 'req-50c054ff-3b69-4e1f-aa67-795d3a1772f3'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:36.462535 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-50c054ff-3b69-4e1f-aa67-795d3a1772f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '546cd66807054a5a8d7a8137e90a4f36', 'request_id': 'req-50c054ff-3b69-4e1f-aa67-795d3a1772f3'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:36.545235 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-50c054ff-3b69-4e1f-aa67-795d3a1772f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '546cd66807054a5a8d7a8137e90a4f36', 'request_id': 'req-50c054ff-3b69-4e1f-aa67-795d3a1772f3'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:36.562398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-50c054ff-3b69-4e1f-aa67-795d3a1772f3 admin admin] The token cache is being invalidate because project 546cd66807054a5a8d7a8137e90a4f36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:13:36.562681 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-50c054ff-3b69-4e1f-aa67-795d3a1772f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:36.565320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-50c054ff-3b69-4e1f-aa67-795d3a1772f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:36.570566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 811/1641] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:13:36 2022] DELETE /identity/v3/projects/546cd66807054a5a8d7a8137e90a4f36 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:36.572301 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a833b87-f6cc-4375-aa2f-42d96092ecdc None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:36.572784 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a833b87-f6cc-4375-aa2f-42d96092ecdc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:36.572876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a833b87-f6cc-4375-aa2f-42d96092ecdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:36.626635 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1a833b87-f6cc-4375-aa2f-42d96092ecdc None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:36.631758 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1a833b87-f6cc-4375-aa2f-42d96092ecdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:36.892747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 831/1642] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:36.894526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fa9eb-0d02-42c6-a141-3335ec274f15 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:36.894813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fa9eb-0d02-42c6-a141-3335ec274f15 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:36.895121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fa9eb-0d02-42c6-a141-3335ec274f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:36.951851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-4e4fa9eb-0d02-42c6-a141-3335ec274f15 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:36.953464 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4e4fa9eb-0d02-42c6-a141-3335ec274f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:36.986435 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 812/1643] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:36.987970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a833b87-f6cc-4375-aa2f-42d96092ecdc None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:37.004009 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a833b87-f6cc-4375-aa2f-42d96092ecdc None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:37.014689 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6d157-075b-41fd-9064-6d649d7a6369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:37.015421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6d157-075b-41fd-9064-6d649d7a6369 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:37.020658 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6d157-075b-41fd-9064-6d649d7a6369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:37.020658 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6d157-075b-41fd-9064-6d649d7a6369 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:37.020658 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6d157-075b-41fd-9064-6d649d7a6369 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:13:37.020658 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6d157-075b-41fd-9064-6d649d7a6369 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:37.020658 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6d157-075b-41fd-9064-6d649d7a6369 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:37.054532 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 832/1644] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:13:37.055736 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fa9eb-0d02-42c6-a141-3335ec274f15 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:37.070607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fa9eb-0d02-42c6-a141-3335ec274f15 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:37.081063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b27e7c-bb54-40d6-8a07-6328a87c337c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:37.081904 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b27e7c-bb54-40d6-8a07-6328a87c337c admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:37.082348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b27e7c-bb54-40d6-8a07-6328a87c337c admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:37.082637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b27e7c-bb54-40d6-8a07-6328a87c337c admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:37.083335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b27e7c-bb54-40d6-8a07-6328a87c337c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:37.092028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b27e7c-bb54-40d6-8a07-6328a87c337c admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:37.248910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 813/1645] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:13:37 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:37.252852 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6d157-075b-41fd-9064-6d649d7a6369 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:37.265480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6d157-075b-41fd-9064-6d649d7a6369 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:37.389916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec089b2-f4d3-4a45-a662-26a214d99301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:37.390426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec089b2-f4d3-4a45-a662-26a214d99301 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:37.390713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec089b2-f4d3-4a45-a662-26a214d99301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:37.391008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec089b2-f4d3-4a45-a662-26a214d99301 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:37.391660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec089b2-f4d3-4a45-a662-26a214d99301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:37.399160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec089b2-f4d3-4a45-a662-26a214d99301 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:37.403028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc4000-c86a-4473-914e-dd7ec8023535 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:37.403236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc4000-c86a-4473-914e-dd7ec8023535 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:37.403524 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc4000-c86a-4473-914e-dd7ec8023535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:37.472004 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-98fc4000-c86a-4473-914e-dd7ec8023535 None None] MFA Rules not processed for user `9427fe8723fb4b298b42106a862a798f`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:37.474017 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-98fc4000-c86a-4473-914e-dd7ec8023535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:37.557426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 833/1646] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:13:37 2022] POST /identity/v3/users => generated 291 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:37.559702 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34af3d-44bc-45cb-8582-857f38d786c9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:37.559984 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34af3d-44bc-45cb-8582-857f38d786c9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:37.560256 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34af3d-44bc-45cb-8582-857f38d786c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:37.588811 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 814/1647] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:13:37.589714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc4000-c86a-4473-914e-dd7ec8023535 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:37.601945 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc4000-c86a-4473-914e-dd7ec8023535 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:37.610299 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a80ec-6faa-4e1b-a7ad-a7695ebd6204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:37.611236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a80ec-6faa-4e1b-a7ad-a7695ebd6204 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:37.611643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a80ec-6faa-4e1b-a7ad-a7695ebd6204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:37.612099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a80ec-6faa-4e1b-a7ad-a7695ebd6204 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:37.612804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a80ec-6faa-4e1b-a7ad-a7695ebd6204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:37.623654 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a80ec-6faa-4e1b-a7ad-a7695ebd6204 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:37.654311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 815/1648] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:37.660451 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3ffa3-8a40-4252-a833-8bb8a4cdeb1b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:37.661590 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9c34af3d-44bc-45cb-8582-857f38d786c9 None None] MFA Rules not processed for user `36b26040882a4e85b82b40a3c17d33bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:37.662346 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3ffa3-8a40-4252-a833-8bb8a4cdeb1b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:37.662721 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3ffa3-8a40-4252-a833-8bb8a4cdeb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:37.663770 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9c34af3d-44bc-45cb-8582-857f38d786c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:37.690320 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:37.713359 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-72b3ffa3-8a40-4252-a833-8bb8a4cdeb1b None None] MFA Rules not processed for user `9427fe8723fb4b298b42106a862a798f`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:37.717348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-72b3ffa3-8a40-4252-a833-8bb8a4cdeb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:37.736356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 816/1649] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:37.739109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3ffa3-8a40-4252-a833-8bb8a4cdeb1b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:37.764778 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3ffa3-8a40-4252-a833-8bb8a4cdeb1b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:37.776861 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b69673-4dfe-4d3e-aa1a-68a5089cbeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:37.777719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b69673-4dfe-4d3e-aa1a-68a5089cbeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:37.779622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b69673-4dfe-4d3e-aa1a-68a5089cbeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:37.780009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b69673-4dfe-4d3e-aa1a-68a5089cbeb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:37.780814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b69673-4dfe-4d3e-aa1a-68a5089cbeb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:37.783294 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b69673-4dfe-4d3e-aa1a-68a5089cbeb3 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:37.788693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:37.807199 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 817/1650] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:37.808989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b69673-4dfe-4d3e-aa1a-68a5089cbeb3 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:37.817740 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f8b69673-4dfe-4d3e-aa1a-68a5089cbeb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:37.838641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 834/1651] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:37.838641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34af3d-44bc-45cb-8582-857f38d786c9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:37.860497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34af3d-44bc-45cb-8582-857f38d786c9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:37.879055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797b08d-003f-46c5-bb5e-ea3b49fdfff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:37.880897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797b08d-003f-46c5-bb5e-ea3b49fdfff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:37.881808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797b08d-003f-46c5-bb5e-ea3b49fdfff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:37.882440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797b08d-003f-46c5-bb5e-ea3b49fdfff3 admin admin] PATH_INFO: `/v3/projects/eafd10a389bb46449029240ec52d3b0c/users/e82bdb1202044a638c3cd0abb19fcc31/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:37.920831 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797b08d-003f-46c5-bb5e-ea3b49fdfff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eafd10a389bb46449029240ec52d3b0c, user_id=e82bdb1202044a638c3cd0abb19fcc31, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:37.927621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797b08d-003f-46c5-bb5e-ea3b49fdfff3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:38.019699 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b69673-4dfe-4d3e-aa1a-68a5089cbeb3 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:38.035788 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3f59d-881d-4da0-aea6-49cf93e3d6d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:38.036684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3f59d-881d-4da0-aea6-49cf93e3d6d4 service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:38.037130 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3f59d-881d-4da0-aea6-49cf93e3d6d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:38.037673 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3f59d-881d-4da0-aea6-49cf93e3d6d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:38.048124 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 835/1652] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:13:37 2022] PUT /identity/v3/projects/eafd10a389bb46449029240ec52d3b0c/users/e82bdb1202044a638c3cd0abb19fcc31/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:38.059013 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3f59d-881d-4da0-aea6-49cf93e3d6d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:38.059791 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797b08d-003f-46c5-bb5e-ea3b49fdfff3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:38.061382 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3f59d-881d-4da0-aea6-49cf93e3d6d4 service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:38.062241 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8797b08d-003f-46c5-bb5e-ea3b49fdfff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:38.085174 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 818/1653] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:38.086679 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6dee1-112b-4e8d-858e-5fd7dea77216 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:38.087149 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6dee1-112b-4e8d-858e-5fd7dea77216 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:38.087582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6dee1-112b-4e8d-858e-5fd7dea77216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:38.172731 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-e5e6dee1-112b-4e8d-858e-5fd7dea77216 None None] MFA Rules not processed for user `cb73bbd331754280a2a0cd64290ec3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:38.173969 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e5e6dee1-112b-4e8d-858e-5fd7dea77216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:38.236583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797b08d-003f-46c5-bb5e-ea3b49fdfff3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:38.248264 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c914bb0-0ff6-4e5c-ae04-aeab045ecfde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:38.249150 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c914bb0-0ff6-4e5c-ae04-aeab045ecfde service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:38.263284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c914bb0-0ff6-4e5c-ae04-aeab045ecfde service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:38.265569 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c914bb0-0ff6-4e5c-ae04-aeab045ecfde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:38.344578 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 819/1654] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:38.346430 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fcdec-adea-441f-bdba-355d8de485b0 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:38.347849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fcdec-adea-441f-bdba-355d8de485b0 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:38.348131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fcdec-adea-441f-bdba-355d8de485b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:38.410829 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c914bb0-0ff6-4e5c-ae04-aeab045ecfde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:38.412664 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c914bb0-0ff6-4e5c-ae04-aeab045ecfde service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:38.435319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 836/1655] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:38.459945 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-1f5fcdec-adea-441f-bdba-355d8de485b0 None None] MFA Rules not processed for user `e82bdb1202044a638c3cd0abb19fcc31`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:38.464809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1f5fcdec-adea-441f-bdba-355d8de485b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:38.719815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c914bb0-0ff6-4e5c-ae04-aeab045ecfde service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:38.727579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 820/1656] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:38.728584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5fcdec-adea-441f-bdba-355d8de485b0 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:38.749804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:38.756264 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c914bb0-0ff6-4e5c-ae04-aeab045ecfde service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:38.774567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5fedf-b9d7-45fd-b55a-12c5b894b5f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:38.775223 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5fedf-b9d7-45fd-b55a-12c5b894b5f7 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:38.775496 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5fedf-b9d7-45fd-b55a-12c5b894b5f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:38.775768 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5fedf-b9d7-45fd-b55a-12c5b894b5f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:38.798839 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5fedf-b9d7-45fd-b55a-12c5b894b5f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:38.812099 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5fedf-b9d7-45fd-b55a-12c5b894b5f7 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:38.858044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 837/1657] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:38.999439 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5fcdec-adea-441f-bdba-355d8de485b0 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:39.021946 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd84df-2aa6-422a-86f8-567f6181b8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:39.024337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd84df-2aa6-422a-86f8-567f6181b8a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:39.025192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd84df-2aa6-422a-86f8-567f6181b8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:39.025315 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd84df-2aa6-422a-86f8-567f6181b8a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:39.027730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd84df-2aa6-422a-86f8-567f6181b8a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:39.039047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd84df-2aa6-422a-86f8-567f6181b8a1 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:39.238558 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 821/1658] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:13:38 2022] POST /identity/v3/projects => generated 379 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:39.242661 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5fedf-b9d7-45fd-b55a-12c5b894b5f7 service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:39.265125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5fedf-b9d7-45fd-b55a-12c5b894b5f7 service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:39.289277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:39.293625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359d420-3476-42b7-a71d-bbb097729e0e None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:39.296850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359d420-3476-42b7-a71d-bbb097729e0e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:39.296850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359d420-3476-42b7-a71d-bbb097729e0e None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:39.296850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 822/1659] 10.222.0.30 () {54 vars in 912 bytes} [Thu Oct 6 08:13:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 06 08:13:39.302510 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c649d4-0907-46d3-8f87-e5c73b0c80a6 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:39.302904 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c649d4-0907-46d3-8f87-e5c73b0c80a6 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:39.303303 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c649d4-0907-46d3-8f87-e5c73b0c80a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:39.384694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-28c649d4-0907-46d3-8f87-e5c73b0c80a6 None None] MFA Rules not processed for user `bf6798f89acf44e897c481924edd9047`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:39.388018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-28c649d4-0907-46d3-8f87-e5c73b0c80a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:39.434583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c478b3-d372-455c-b3dd-65f608f2fb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:39.435245 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c478b3-d372-455c-b3dd-65f608f2fb75 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:39.444012 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c478b3-d372-455c-b3dd-65f608f2fb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:39.444012 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c478b3-d372-455c-b3dd-65f608f2fb75 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:39.444012 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c478b3-d372-455c-b3dd-65f608f2fb75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:39.444012 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c478b3-d372-455c-b3dd-65f608f2fb75 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:39.531429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 823/1660] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Oct 6 08:13:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:13:39.534600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c649d4-0907-46d3-8f87-e5c73b0c80a6 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:39.536888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-28c649d4-0907-46d3-8f87-e5c73b0c80a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:39.552083 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c649d4-0907-46d3-8f87-e5c73b0c80a6 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:39.576570 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea715b8c-b39a-4f89-b02c-a4979ba5db2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:39.576570 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea715b8c-b39a-4f89-b02c-a4979ba5db2e service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:39.576570 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea715b8c-b39a-4f89-b02c-a4979ba5db2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:39.576570 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea715b8c-b39a-4f89-b02c-a4979ba5db2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:39.594299 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea715b8c-b39a-4f89-b02c-a4979ba5db2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:39.599532 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea715b8c-b39a-4f89-b02c-a4979ba5db2e service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:39.611265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 838/1661] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:13:39 2022] POST /identity/v3/users => generated 296 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:39.612981 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c478b3-d372-455c-b3dd-65f608f2fb75 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:39.613264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 824/1662] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:13:39.642605 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c478b3-d372-455c-b3dd-65f608f2fb75 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:39.653570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a1ee8-db55-4ab6-bda1-689701f2d2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:39.654435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a1ee8-db55-4ab6-bda1-689701f2d2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:39.655736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a1ee8-db55-4ab6-bda1-689701f2d2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:39.656637 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a1ee8-db55-4ab6-bda1-689701f2d2b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:39.659009 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a1ee8-db55-4ab6-bda1-689701f2d2b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:39.664351 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a1ee8-db55-4ab6-bda1-689701f2d2b0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:39.699320 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 839/1663] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:39.701178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea715b8c-b39a-4f89-b02c-a4979ba5db2e service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:39.730046 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea715b8c-b39a-4f89-b02c-a4979ba5db2e service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:39.757735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed892ad-ef87-4acc-8d02-872ce7b7dc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:39.764086 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed892ad-ef87-4acc-8d02-872ce7b7dc23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:39.765669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed892ad-ef87-4acc-8d02-872ce7b7dc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:39.772905 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed892ad-ef87-4acc-8d02-872ce7b7dc23 admin admin] PATH_INFO: `/v3/projects/6e8e833656e0447081bfd873d988cb43/users/46343636bb2c4f9e9050c4d074485a98/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:39.849312 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed892ad-ef87-4acc-8d02-872ce7b7dc23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8e833656e0447081bfd873d988cb43, user_id=46343636bb2c4f9e9050c4d074485a98, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:39.860948 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed892ad-ef87-4acc-8d02-872ce7b7dc23 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:39.965440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 825/1664] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:13:39 2022] PUT /identity/v3/projects/6e8e833656e0447081bfd873d988cb43/users/46343636bb2c4f9e9050c4d074485a98/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:39.966189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a1ee8-db55-4ab6-bda1-689701f2d2b0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:39.979110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a1ee8-db55-4ab6-bda1-689701f2d2b0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:40.224159 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d67f6-1785-445c-84b1-789a8eeb7b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:40.224851 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d67f6-1785-445c-84b1-789a8eeb7b7b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:40.225122 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d67f6-1785-445c-84b1-789a8eeb7b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:40.230034 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d67f6-1785-445c-84b1-789a8eeb7b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:40.230034 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d67f6-1785-445c-84b1-789a8eeb7b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:40.232073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d67f6-1785-445c-84b1-789a8eeb7b7b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:40.282410 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed892ad-ef87-4acc-8d02-872ce7b7dc23 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:40.290220 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 840/1665] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:40.299799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed892ad-ef87-4acc-8d02-872ce7b7dc23 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:40.316953 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8eedd-6288-4c1b-ad92-3d34640843eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:40.318978 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8eedd-6288-4c1b-ad92-3d34640843eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:40.319442 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8eedd-6288-4c1b-ad92-3d34640843eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:40.334119 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8eedd-6288-4c1b-ad92-3d34640843eb admin admin] PATH_INFO: `/v3/projects/6e8e833656e0447081bfd873d988cb43/users/46343636bb2c4f9e9050c4d074485a98/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:40.361250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8eedd-6288-4c1b-ad92-3d34640843eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8e833656e0447081bfd873d988cb43, user_id=46343636bb2c4f9e9050c4d074485a98, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:40.366864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8eedd-6288-4c1b-ad92-3d34640843eb admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:40.402050 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bbbb3-9a4e-498b-801f-782bea3629a4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:40.402330 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bbbb3-9a4e-498b-801f-782bea3629a4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:40.403914 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bbbb3-9a4e-498b-801f-782bea3629a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:40.408493 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 826/1666] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:13:40 2022] PUT /identity/v3/projects/6e8e833656e0447081bfd873d988cb43/users/46343636bb2c4f9e9050c4d074485a98/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:13:40.465310 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-577bbbb3-9a4e-498b-801f-782bea3629a4 None None] MFA Rules not processed for user `46343636bb2c4f9e9050c4d074485a98`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:40.467933 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-577bbbb3-9a4e-498b-801f-782bea3629a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:40.683296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 841/1667] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:40 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:13:40.752954 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8eedd-6288-4c1b-ad92-3d34640843eb admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:40.755072 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-51f8eedd-6288-4c1b-ad92-3d34640843eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:40.896081 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:40.977383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8eedd-6288-4c1b-ad92-3d34640843eb admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:40.991196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c00be-d737-4739-b564-7a0b8dd910d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:40.992593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c00be-d737-4739-b564-7a0b8dd910d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:40.998400 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c00be-d737-4739-b564-7a0b8dd910d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:40.998889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c00be-d737-4739-b564-7a0b8dd910d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:41.023934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c00be-d737-4739-b564-7a0b8dd910d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:41.026182 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c00be-d737-4739-b564-7a0b8dd910d9 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:41.070885 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 827/1668] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:41.199184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bbbb3-9a4e-498b-801f-782bea3629a4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:41.200670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-577bbbb3-9a4e-498b-801f-782bea3629a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:41.245982 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:41.505025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bbbb3-9a4e-498b-801f-782bea3629a4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:41.532766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64509d5-118a-493f-b669-eead4d0205ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6074872303d3468fb566378fa34e8bea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:41.534208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64509d5-118a-493f-b669-eead4d0205ba service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:41.534486 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64509d5-118a-493f-b669-eead4d0205ba service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:41.534756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64509d5-118a-493f-b669-eead4d0205ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:41.541492 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c64509d5-118a-493f-b669-eead4d0205ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:41.575579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64509d5-118a-493f-b669-eead4d0205ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:41.577766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64509d5-118a-493f-b669-eead4d0205ba service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:41.608029 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c00be-d737-4739-b564-7a0b8dd910d9 service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:41.611937 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d85c00be-d737-4739-b564-7a0b8dd910d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:41.615941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 842/1669] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:41.638392 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c00be-d737-4739-b564-7a0b8dd910d9 service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:41.655988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de809ace-e02c-4c75-a6a8-93f5f215485c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:41.658304 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de809ace-e02c-4c75-a6a8-93f5f215485c service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:41.659673 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de809ace-e02c-4c75-a6a8-93f5f215485c service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:41.660936 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de809ace-e02c-4c75-a6a8-93f5f215485c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:41.840679 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de809ace-e02c-4c75-a6a8-93f5f215485c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:41.844029 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de809ace-e02c-4c75-a6a8-93f5f215485c service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:41.867184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 828/1670] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:44.626783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242492bf-75f2-424c-b810-f13fecfc4622 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:44.627778 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a092c2-43f8-49ff-a514-25a6156d7749 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:44.628859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242492bf-75f2-424c-b810-f13fecfc4622 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:44.629423 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a092c2-43f8-49ff-a514-25a6156d7749 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:44.629883 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a092c2-43f8-49ff-a514-25a6156d7749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:44.630353 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242492bf-75f2-424c-b810-f13fecfc4622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:44.748041 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-14a092c2-43f8-49ff-a514-25a6156d7749 None None] MFA Rules not processed for user `85bdfc4b6ad443d68ebc0d6932ccad66`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:44.749278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-14a092c2-43f8-49ff-a514-25a6156d7749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:44.752394 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-242492bf-75f2-424c-b810-f13fecfc4622 None None] MFA Rules not processed for user `9932c7fb57394d709a27023f3ae2677e`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:44.754321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-242492bf-75f2-424c-b810-f13fecfc4622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:44.989607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 829/1672] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:44.999600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92eedf0-85d7-4edf-944b-99d6c5ab8c15 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:44.999862 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 843/1672] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:45.000305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92eedf0-85d7-4edf-944b-99d6c5ab8c15 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:45.012143 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92eedf0-85d7-4edf-944b-99d6c5ab8c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:45.013343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a092c2-43f8-49ff-a514-25a6156d7749 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:45.057476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-a92eedf0-85d7-4edf-944b-99d6c5ab8c15 None None] MFA Rules not processed for user `85bdfc4b6ad443d68ebc0d6932ccad66`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:45.058718 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a92eedf0-85d7-4edf-944b-99d6c5ab8c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:45.089129 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 830/1673] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:45.103124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92eedf0-85d7-4edf-944b-99d6c5ab8c15 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:45.236235 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a092c2-43f8-49ff-a514-25a6156d7749 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:45.279984 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d0100-6ce5-4e42-8812-6fc43b93b12a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:45.286552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d0100-6ce5-4e42-8812-6fc43b93b12a service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:45.286552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d0100-6ce5-4e42-8812-6fc43b93b12a service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:45.286552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d0100-6ce5-4e42-8812-6fc43b93b12a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:45.318596 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d0100-6ce5-4e42-8812-6fc43b93b12a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:45.318987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:45.322501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d0100-6ce5-4e42-8812-6fc43b93b12a service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:45.347907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 844/1674] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:45.355942 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd028a3-3f33-4fb1-b1f4-15fb72720023 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:45.356182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd028a3-3f33-4fb1-b1f4-15fb72720023 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:45.357314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd028a3-3f33-4fb1-b1f4-15fb72720023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:45.362030 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92eedf0-85d7-4edf-944b-99d6c5ab8c15 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:45.375330 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7663b-8474-4695-a4b4-613049d033c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:45.378847 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7663b-8474-4695-a4b4-613049d033c6 service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:45.379316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7663b-8474-4695-a4b4-613049d033c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:45.379730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7663b-8474-4695-a4b4-613049d033c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:45.404153 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7663b-8474-4695-a4b4-613049d033c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:45.406147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7663b-8474-4695-a4b4-613049d033c6 service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:45.472440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 831/1675] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:13:45.480539 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1bd028a3-3f33-4fb1-b1f4-15fb72720023 None None] MFA Rules not processed for user `36b26040882a4e85b82b40a3c17d33bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:45.507981 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1bd028a3-3f33-4fb1-b1f4-15fb72720023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:45.534372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:45.642435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 845/1676] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:46.998518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7663b-8474-4695-a4b4-613049d033c6 service nova] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:47.000171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e4a7663b-8474-4695-a4b4-613049d033c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:47.202505 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98640228-ca55-42e2-9b8e-2b814155ef9b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:47.202977 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98640228-ca55-42e2-9b8e-2b814155ef9b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:47.203507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98640228-ca55-42e2-9b8e-2b814155ef9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:47.246760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7663b-8474-4695-a4b4-613049d033c6 service nova] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:47.265345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-98640228-ca55-42e2-9b8e-2b814155ef9b None None] MFA Rules not processed for user `cb73bbd331754280a2a0cd64290ec3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:47.266651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f7799-8f5f-4922-b762-8a1b73bc737d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:47.268294 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7799-8f5f-4922-b762-8a1b73bc737d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:47.271111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-98640228-ca55-42e2-9b8e-2b814155ef9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:47.272661 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7799-8f5f-4922-b762-8a1b73bc737d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:47.273059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7799-8f5f-4922-b762-8a1b73bc737d admin admin] PATH_INFO: `/v3/users/9180d83ed50f4236b25c5cb73f478686` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:47.278443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f7799-8f5f-4922-b762-8a1b73bc737d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9180d83ed50f4236b25c5cb73f478686)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:47.284187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: WARNING py.warnings [None req-669f7799-8f5f-4922-b762-8a1b73bc737d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 08:13:47.284187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: warnings.warn(msg) Oct 06 08:13:47.284187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: Oct 06 08:13:47.296635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f7799-8f5f-4922-b762-8a1b73bc737d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:47.491035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 846/1677] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:13:47.493765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6613627-243f-492c-9ea4-137270862e15 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:47.499596 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6613627-243f-492c-9ea4-137270862e15 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:47.499987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6613627-243f-492c-9ea4-137270862e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:47.632696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e6613627-243f-492c-9ea4-137270862e15 None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:47.636894 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e6613627-243f-492c-9ea4-137270862e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:47.679399 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-669f7799-8f5f-4922-b762-8a1b73bc737d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9180d83ed50f4236b25c5cb73f478686'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:47.680124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-669f7799-8f5f-4922-b762-8a1b73bc737d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9180d83ed50f4236b25c5cb73f478686'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:47.758154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-669f7799-8f5f-4922-b762-8a1b73bc737d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9180d83ed50f4236b25c5cb73f478686'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:47.779573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 847/1678] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Oct 6 08:13:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:47.884692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-669f7799-8f5f-4922-b762-8a1b73bc737d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9180d83ed50f4236b25c5cb73f478686'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:47.886966 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-669f7799-8f5f-4922-b762-8a1b73bc737d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9180d83ed50f4236b25c5cb73f478686'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:47.939195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 832/1679] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:13:46 2022] DELETE /identity/v3/users/9180d83ed50f4236b25c5cb73f478686 => generated 0 bytes in 939 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:47.942894 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6699aa-f1ff-4ccc-9836-062efb5c1802 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:47.943587 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6699aa-f1ff-4ccc-9836-062efb5c1802 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:47.943977 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6699aa-f1ff-4ccc-9836-062efb5c1802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:47.992011 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec60e3f-6074-4057-9dd0-ea35f9104325 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:47.992294 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec60e3f-6074-4057-9dd0-ea35f9104325 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:47.992874 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec60e3f-6074-4057-9dd0-ea35f9104325 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:47.994209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 833/1680] 10.222.0.30 () {54 vars in 912 bytes} [Thu Oct 6 08:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:13:48.001621 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af3f3a-c612-4525-9178-5b530bdaabaa None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:48.002228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af3f3a-c612-4525-9178-5b530bdaabaa None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:48.002228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af3f3a-c612-4525-9178-5b530bdaabaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:48.006579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fa6699aa-f1ff-4ccc-9836-062efb5c1802 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:48.007409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fa6699aa-f1ff-4ccc-9836-062efb5c1802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:48.026343 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-d9af3f3a-c612-4525-9178-5b530bdaabaa None None] MFA Rules not processed for user `bf6798f89acf44e897c481924edd9047`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:48.027876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d9af3f3a-c612-4525-9178-5b530bdaabaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:48.132903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:48.176282 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 848/1681] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:13:48.177941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728efe6-dc45-490e-98be-50d0670340c0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:48.178463 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728efe6-dc45-490e-98be-50d0670340c0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:48.178991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728efe6-dc45-490e-98be-50d0670340c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:48.209090 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6728efe6-dc45-490e-98be-50d0670340c0 None None] MFA Rules not processed for user `85bdfc4b6ad443d68ebc0d6932ccad66`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:48.211822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6728efe6-dc45-490e-98be-50d0670340c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:48.241544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 834/1682] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Oct 6 08:13:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:13:48.242639 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af3f3a-c612-4525-9178-5b530bdaabaa None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:48.249900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d9af3f3a-c612-4525-9178-5b530bdaabaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:48.274631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af3f3a-c612-4525-9178-5b530bdaabaa None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:48.281381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59a7de-5439-4e2c-95f3-36b8545b7230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:48.282183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59a7de-5439-4e2c-95f3-36b8545b7230 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:48.282694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59a7de-5439-4e2c-95f3-36b8545b7230 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:48.283135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59a7de-5439-4e2c-95f3-36b8545b7230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:48.320226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59a7de-5439-4e2c-95f3-36b8545b7230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:48.323696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59a7de-5439-4e2c-95f3-36b8545b7230 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:48.340363 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:48.354971 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 835/1683] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:48.356082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59a7de-5439-4e2c-95f3-36b8545b7230 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:48.376525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 849/1684] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:48.383214 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59a7de-5439-4e2c-95f3-36b8545b7230 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:48.395654 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770e23f-e8b5-42e8-a4d3-9658b39c78fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:48.396729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770e23f-e8b5-42e8-a4d3-9658b39c78fa service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:48.397157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770e23f-e8b5-42e8-a4d3-9658b39c78fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:48.397576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770e23f-e8b5-42e8-a4d3-9658b39c78fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:48.413159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770e23f-e8b5-42e8-a4d3-9658b39c78fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:48.414882 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770e23f-e8b5-42e8-a4d3-9658b39c78fa service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:48.435374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728efe6-dc45-490e-98be-50d0670340c0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:48.437594 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6728efe6-dc45-490e-98be-50d0670340c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:48.441002 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 836/1685] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:48.778848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728efe6-dc45-490e-98be-50d0670340c0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:48.801610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ef868-66d2-462d-8c55-a5e70e9d01da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:48.802260 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ef868-66d2-462d-8c55-a5e70e9d01da service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:48.802534 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ef868-66d2-462d-8c55-a5e70e9d01da service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:48.802801 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ef868-66d2-462d-8c55-a5e70e9d01da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:49.015382 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ef868-66d2-462d-8c55-a5e70e9d01da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:49.017270 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ef868-66d2-462d-8c55-a5e70e9d01da service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:49.039502 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 850/1686] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:13:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:13:51.002116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770e23f-e8b5-42e8-a4d3-9658b39c78fa service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:51.004664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f770e23f-e8b5-42e8-a4d3-9658b39c78fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:51.039115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770e23f-e8b5-42e8-a4d3-9658b39c78fa service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:51.077193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99c48e-6aa4-408d-81f3-fdf581aba164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:51.078446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99c48e-6aa4-408d-81f3-fdf581aba164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:51.079924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99c48e-6aa4-408d-81f3-fdf581aba164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:51.084337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99c48e-6aa4-408d-81f3-fdf581aba164 admin admin] PATH_INFO: `/v3/projects/74ade644bb9a4efe8ebb365a3d076522` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:51.086689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99c48e-6aa4-408d-81f3-fdf581aba164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74ade644bb9a4efe8ebb365a3d076522)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:51.098046 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99c48e-6aa4-408d-81f3-fdf581aba164 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:51.362185 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-ab99c48e-6aa4-408d-81f3-fdf581aba164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74ade644bb9a4efe8ebb365a3d076522', 'request_id': 'req-ab99c48e-6aa4-408d-81f3-fdf581aba164'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:51.384519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-ab99c48e-6aa4-408d-81f3-fdf581aba164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74ade644bb9a4efe8ebb365a3d076522', 'request_id': 'req-ab99c48e-6aa4-408d-81f3-fdf581aba164'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:51.509209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-ab99c48e-6aa4-408d-81f3-fdf581aba164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74ade644bb9a4efe8ebb365a3d076522', 'request_id': 'req-ab99c48e-6aa4-408d-81f3-fdf581aba164'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:51.527032 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-ab99c48e-6aa4-408d-81f3-fdf581aba164 admin admin] The token cache is being invalidate because project 74ade644bb9a4efe8ebb365a3d076522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:13:51.527242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-ab99c48e-6aa4-408d-81f3-fdf581aba164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:51.527943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-ab99c48e-6aa4-408d-81f3-fdf581aba164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:51.531025 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 837/1687] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:13:51 2022] DELETE /identity/v3/projects/74ade644bb9a4efe8ebb365a3d076522 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:51.535254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bceda-799b-48c6-93af-16439409e447 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:51.536960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bceda-799b-48c6-93af-16439409e447 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:51.537410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bceda-799b-48c6-93af-16439409e447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:51.571442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:51.605726 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b26bceda-799b-48c6-93af-16439409e447 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:51.607939 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b26bceda-799b-48c6-93af-16439409e447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:51.831381 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 851/1688] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:51.844764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7675a-b9bc-4270-91c1-df16a21407b0 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:51.844764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7675a-b9bc-4270-91c1-df16a21407b0 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:51.844764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7675a-b9bc-4270-91c1-df16a21407b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:51.887758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-00e7675a-b9bc-4270-91c1-df16a21407b0 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:51.887758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-00e7675a-b9bc-4270-91c1-df16a21407b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:51.899383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 838/1689] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:51.907799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bceda-799b-48c6-93af-16439409e447 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:51.921780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bceda-799b-48c6-93af-16439409e447 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:51.930917 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f18fb9-26ae-4b96-80f4-d8aa176d278c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:51.931593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f18fb9-26ae-4b96-80f4-d8aa176d278c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:51.932381 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f18fb9-26ae-4b96-80f4-d8aa176d278c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:51.932753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f18fb9-26ae-4b96-80f4-d8aa176d278c admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:51.933429 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f18fb9-26ae-4b96-80f4-d8aa176d278c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:13:51.933839 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f18fb9-26ae-4b96-80f4-d8aa176d278c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:51.937624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f18fb9-26ae-4b96-80f4-d8aa176d278c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:51.953545 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 852/1690] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:13:51.957993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7675a-b9bc-4270-91c1-df16a21407b0 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:51.969753 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7675a-b9bc-4270-91c1-df16a21407b0 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:51.977624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23fde30-ba30-4c3d-abea-bb54d7bc1c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:51.978258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fde30-ba30-4c3d-abea-bb54d7bc1c5e admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:51.978538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fde30-ba30-4c3d-abea-bb54d7bc1c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:51.979407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fde30-ba30-4c3d-abea-bb54d7bc1c5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:51.980124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23fde30-ba30-4c3d-abea-bb54d7bc1c5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:51.982572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23fde30-ba30-4c3d-abea-bb54d7bc1c5e admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:52.118797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 839/1691] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:13:51 2022] POST /identity/v3/projects => generated 403 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:52.123323 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f18fb9-26ae-4b96-80f4-d8aa176d278c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:52.134124 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f18fb9-26ae-4b96-80f4-d8aa176d278c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:52.170663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb88a51-4d28-46e7-b83d-b2c572f3b98e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:52.170976 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb88a51-4d28-46e7-b83d-b2c572f3b98e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:52.171247 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb88a51-4d28-46e7-b83d-b2c572f3b98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:52.228057 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-5cb88a51-4d28-46e7-b83d-b2c572f3b98e None None] MFA Rules not processed for user `9932c7fb57394d709a27023f3ae2677e`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:52.230909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5cb88a51-4d28-46e7-b83d-b2c572f3b98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:52.263738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f66710-b122-4c01-8bff-9d17e55e6669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:52.264537 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f66710-b122-4c01-8bff-9d17e55e6669 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:52.264762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f66710-b122-4c01-8bff-9d17e55e6669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:52.265046 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f66710-b122-4c01-8bff-9d17e55e6669 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:52.265705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f66710-b122-4c01-8bff-9d17e55e6669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:52.270011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f66710-b122-4c01-8bff-9d17e55e6669 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:52.418046 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 853/1692] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:13:52 2022] POST /identity/v3/users => generated 302 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:52.419202 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f66710-b122-4c01-8bff-9d17e55e6669 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:52.430550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 840/1693] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:13:52.469314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f66710-b122-4c01-8bff-9d17e55e6669 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:52.483109 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6de6f6-997a-418d-953e-d97643a2732e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:52.511291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6de6f6-997a-418d-953e-d97643a2732e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:52.512001 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6de6f6-997a-418d-953e-d97643a2732e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:52.513527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6de6f6-997a-418d-953e-d97643a2732e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:52.514647 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6de6f6-997a-418d-953e-d97643a2732e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:52.523673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6de6f6-997a-418d-953e-d97643a2732e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:52.564611 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 854/1694] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:52.566983 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb88a51-4d28-46e7-b83d-b2c572f3b98e None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:52.590634 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb88a51-4d28-46e7-b83d-b2c572f3b98e None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:52.616771 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b723f94-b70b-4dd3-8bce-4b6a92e31991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:52.619020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b723f94-b70b-4dd3-8bce-4b6a92e31991 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:52.622671 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b723f94-b70b-4dd3-8bce-4b6a92e31991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:52.623139 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b723f94-b70b-4dd3-8bce-4b6a92e31991 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:52.625524 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b723f94-b70b-4dd3-8bce-4b6a92e31991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:52.633514 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b723f94-b70b-4dd3-8bce-4b6a92e31991 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:52.673050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 841/1695] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:13:52.674622 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6de6f6-997a-418d-953e-d97643a2732e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:52.677837 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23825e07-bc3c-4c61-93ab-c41cf17aedea None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:52.678223 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23825e07-bc3c-4c61-93ab-c41cf17aedea None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:52.678785 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23825e07-bc3c-4c61-93ab-c41cf17aedea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:52.692062 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6de6f6-997a-418d-953e-d97643a2732e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:52.709562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a4cf0-9387-460e-af4a-40c77dba9713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:52.715173 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a4cf0-9387-460e-af4a-40c77dba9713 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:52.719155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a4cf0-9387-460e-af4a-40c77dba9713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:52.719837 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a4cf0-9387-460e-af4a-40c77dba9713 admin admin] PATH_INFO: `/v3/projects/b095553ea6ea41c38d5e712f5b93ff32/users/e595ed4897464b6d979d9629181ba81c/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:52.773888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-23825e07-bc3c-4c61-93ab-c41cf17aedea None None] MFA Rules not processed for user `bf6798f89acf44e897c481924edd9047`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:52.806393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-23825e07-bc3c-4c61-93ab-c41cf17aedea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:52.828065 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a4cf0-9387-460e-af4a-40c77dba9713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b095553ea6ea41c38d5e712f5b93ff32, user_id=e595ed4897464b6d979d9629181ba81c, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:52.835701 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a4cf0-9387-460e-af4a-40c77dba9713 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:52.874791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:52.964063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 842/1696] 10.222.0.43 () {58 vars in 1015 bytes} [Thu Oct 6 08:13:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:53.013844 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23825e07-bc3c-4c61-93ab-c41cf17aedea None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:53.016242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-23825e07-bc3c-4c61-93ab-c41cf17aedea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:53.036050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23825e07-bc3c-4c61-93ab-c41cf17aedea None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:53.043958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 855/1697] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:13:52 2022] PUT /identity/v3/projects/b095553ea6ea41c38d5e712f5b93ff32/users/e595ed4897464b6d979d9629181ba81c/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:53.065280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9625bf6-92a6-47f0-8cf7-b3ada65d38c8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:53.065845 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9625bf6-92a6-47f0-8cf7-b3ada65d38c8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:53.066420 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9625bf6-92a6-47f0-8cf7-b3ada65d38c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:53.203225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f9625bf6-92a6-47f0-8cf7-b3ada65d38c8 None None] MFA Rules not processed for user `e595ed4897464b6d979d9629181ba81c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:53.204543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f9625bf6-92a6-47f0-8cf7-b3ada65d38c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:53.324242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187574c-304e-4e67-bbe5-75a42d25e0fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:53.325621 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187574c-304e-4e67-bbe5-75a42d25e0fe service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:53.326402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187574c-304e-4e67-bbe5-75a42d25e0fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:53.326876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187574c-304e-4e67-bbe5-75a42d25e0fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:53.349210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f187574c-304e-4e67-bbe5-75a42d25e0fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:53.351232 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f187574c-304e-4e67-bbe5-75a42d25e0fe service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:53.410547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 843/1698] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:13:53.459256 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:53.605797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 856/1699] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:53 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 541 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:53.610318 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187574c-304e-4e67-bbe5-75a42d25e0fe service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:53.873011 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187574c-304e-4e67-bbe5-75a42d25e0fe service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:53.884531 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb64068-f8ab-4915-ae56-accbdbbb9892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:53.885539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb64068-f8ab-4915-ae56-accbdbbb9892 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:53.885730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb64068-f8ab-4915-ae56-accbdbbb9892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:53.886009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb64068-f8ab-4915-ae56-accbdbbb9892 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:53.886710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb64068-f8ab-4915-ae56-accbdbbb9892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:53.888823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb64068-f8ab-4915-ae56-accbdbbb9892 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:54.013176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 844/1700] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:13:53 2022] POST /identity/v3/projects => generated 405 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:54.019704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9625bf6-92a6-47f0-8cf7-b3ada65d38c8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:54.048427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9625bf6-92a6-47f0-8cf7-b3ada65d38c8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:54.270020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca97af-7046-49ac-807b-ef6fa249f69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:54.270020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca97af-7046-49ac-807b-ef6fa249f69f admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:54.270020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca97af-7046-49ac-807b-ef6fa249f69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:54.270020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca97af-7046-49ac-807b-ef6fa249f69f admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:54.273465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca97af-7046-49ac-807b-ef6fa249f69f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:54.273465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca97af-7046-49ac-807b-ef6fa249f69f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:54.502028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb64068-f8ab-4915-ae56-accbdbbb9892 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:54.515815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 857/1701] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:13:54 2022] POST /identity/v3/users => generated 309 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:13:54.523497 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb64068-f8ab-4915-ae56-accbdbbb9892 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:54.534441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d811cd2-5d10-4ca3-a094-985b9efb1de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:54.535859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d811cd2-5d10-4ca3-a094-985b9efb1de6 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:54.539896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d811cd2-5d10-4ca3-a094-985b9efb1de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:54.540394 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d811cd2-5d10-4ca3-a094-985b9efb1de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:54.561304 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d811cd2-5d10-4ca3-a094-985b9efb1de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:54.564543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d811cd2-5d10-4ca3-a094-985b9efb1de6 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:54.591565 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 845/1702] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:13:54.595791 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca97af-7046-49ac-807b-ef6fa249f69f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:54.621242 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca97af-7046-49ac-807b-ef6fa249f69f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:54.633892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900f3ee-9a85-4e37-bba9-a57f23f72f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:54.634731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900f3ee-9a85-4e37-bba9-a57f23f72f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:54.635651 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900f3ee-9a85-4e37-bba9-a57f23f72f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:54.635926 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900f3ee-9a85-4e37-bba9-a57f23f72f38 admin admin] PATH_INFO: `/v3/projects/6fb9c90ca9b14e939f3c6744555f9f4d/users/834b0c5265414b05bf2a5d683eee69e9/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:54.717181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900f3ee-9a85-4e37-bba9-a57f23f72f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb9c90ca9b14e939f3c6744555f9f4d, user_id=834b0c5265414b05bf2a5d683eee69e9, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:54.725620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900f3ee-9a85-4e37-bba9-a57f23f72f38 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:54.945208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 858/1703] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:13:54 2022] PUT /identity/v3/projects/6fb9c90ca9b14e939f3c6744555f9f4d/users/834b0c5265414b05bf2a5d683eee69e9/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:54.949435 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d811cd2-5d10-4ca3-a094-985b9efb1de6 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:54.964543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d811cd2-5d10-4ca3-a094-985b9efb1de6 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:55.247710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51598f9b-45bf-47b3-8ed9-77c049836213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:55.254796 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598f9b-45bf-47b3-8ed9-77c049836213 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:55.254796 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598f9b-45bf-47b3-8ed9-77c049836213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:55.254796 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598f9b-45bf-47b3-8ed9-77c049836213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:55.254796 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51598f9b-45bf-47b3-8ed9-77c049836213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:55.258863 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51598f9b-45bf-47b3-8ed9-77c049836213 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:55.294127 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 846/1704] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:13:55.295787 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900f3ee-9a85-4e37-bba9-a57f23f72f38 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:55.319443 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900f3ee-9a85-4e37-bba9-a57f23f72f38 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:55.326596 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3f4a9-7867-4465-a959-c4943f343d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:55.327684 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3f4a9-7867-4465-a959-c4943f343d8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:55.334492 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3f4a9-7867-4465-a959-c4943f343d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:55.334849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3f4a9-7867-4465-a959-c4943f343d8c admin admin] PATH_INFO: `/v3/projects/6fb9c90ca9b14e939f3c6744555f9f4d/users/834b0c5265414b05bf2a5d683eee69e9/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:55.357420 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3f4a9-7867-4465-a959-c4943f343d8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb9c90ca9b14e939f3c6744555f9f4d, user_id=834b0c5265414b05bf2a5d683eee69e9, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:55.370569 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3f4a9-7867-4465-a959-c4943f343d8c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:55.458367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 859/1705] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:13:55 2022] PUT /identity/v3/projects/6fb9c90ca9b14e939f3c6744555f9f4d/users/834b0c5265414b05bf2a5d683eee69e9/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:13:55.474192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42441bf-02c8-4132-9f47-40e16cc6ba30 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:55.474308 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42441bf-02c8-4132-9f47-40e16cc6ba30 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:55.474575 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42441bf-02c8-4132-9f47-40e16cc6ba30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:55.571109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-e42441bf-02c8-4132-9f47-40e16cc6ba30 None None] MFA Rules not processed for user `834b0c5265414b05bf2a5d683eee69e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:55.575043 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e42441bf-02c8-4132-9f47-40e16cc6ba30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:55.826894 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 847/1706] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:55 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:13:55.840720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3f4a9-7867-4465-a959-c4943f343d8c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:55.842520 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6ce3f4a9-7867-4465-a959-c4943f343d8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:56.037126 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3f4a9-7867-4465-a959-c4943f343d8c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:56.044243 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722b97e-f3aa-474c-ac9c-ec522857f97c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:56.045364 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722b97e-f3aa-474c-ac9c-ec522857f97c service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:56.045907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722b97e-f3aa-474c-ac9c-ec522857f97c service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:56.046321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722b97e-f3aa-474c-ac9c-ec522857f97c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:56.052123 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:56.064443 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722b97e-f3aa-474c-ac9c-ec522857f97c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:56.067706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722b97e-f3aa-474c-ac9c-ec522857f97c service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:56.084959 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 860/1707] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:55 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:56.457329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42441bf-02c8-4132-9f47-40e16cc6ba30 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:56.811636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42441bf-02c8-4132-9f47-40e16cc6ba30 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:56.851999 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70339d94-b1d7-4b7d-8e61-22807a113787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:56.852421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70339d94-b1d7-4b7d-8e61-22807a113787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:56.852707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70339d94-b1d7-4b7d-8e61-22807a113787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:56.852989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70339d94-b1d7-4b7d-8e61-22807a113787 admin admin] PATH_INFO: `/v3/users/e595ed4897464b6d979d9629181ba81c` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:56.859937 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70339d94-b1d7-4b7d-8e61-22807a113787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e595ed4897464b6d979d9629181ba81c)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:56.862751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70339d94-b1d7-4b7d-8e61-22807a113787 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:57.241902 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b012c-9d0d-45df-b327-dc4a299bd652 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:57.242782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b012c-9d0d-45df-b327-dc4a299bd652 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:57.243224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b012c-9d0d-45df-b327-dc4a299bd652 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:57.245295 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 861/1708] 10.222.0.30 () {54 vars in 921 bytes} [Thu Oct 6 08:13:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:13:57.284714 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ef1d4-e3a6-4dff-bc05-7f3dc21debbf None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:57.285240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ef1d4-e3a6-4dff-bc05-7f3dc21debbf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:57.285776 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ef1d4-e3a6-4dff-bc05-7f3dc21debbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:57.372134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-769ef1d4-e3a6-4dff-bc05-7f3dc21debbf None None] MFA Rules not processed for user `209795285384486183cda455fe316bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:57.387095 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-769ef1d4-e3a6-4dff-bc05-7f3dc21debbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:57.444794 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:57.547462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-70339d94-b1d7-4b7d-8e61-22807a113787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e595ed4897464b6d979d9629181ba81c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:57.549115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-70339d94-b1d7-4b7d-8e61-22807a113787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e595ed4897464b6d979d9629181ba81c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:57.575576 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 862/1709] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Oct 6 08:13:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:13:57.594011 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-70339d94-b1d7-4b7d-8e61-22807a113787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e595ed4897464b6d979d9629181ba81c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:57.724429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-70339d94-b1d7-4b7d-8e61-22807a113787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e595ed4897464b6d979d9629181ba81c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:57.726003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-70339d94-b1d7-4b7d-8e61-22807a113787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e595ed4897464b6d979d9629181ba81c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:13:57.769472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 848/1710] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:13:56 2022] DELETE /identity/v3/users/e595ed4897464b6d979d9629181ba81c => generated 0 bytes in 1314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:13:57.774161 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5624c9-ca09-4a77-b988-5c967ef3cd97 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:57.774901 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5624c9-ca09-4a77-b988-5c967ef3cd97 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:57.777115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5624c9-ca09-4a77-b988-5c967ef3cd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:57.804975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70339d94-b1d7-4b7d-8e61-22807a113787 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:57.806627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-70339d94-b1d7-4b7d-8e61-22807a113787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:57.847631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ae5624c9-ca09-4a77-b988-5c967ef3cd97 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:13:57.850575 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ae5624c9-ca09-4a77-b988-5c967ef3cd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:58.037232 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 863/1711] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:13:58.048547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5624c9-ca09-4a77-b988-5c967ef3cd97 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:13:58.052117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ae5624c9-ca09-4a77-b988-5c967ef3cd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:13:58.190590 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70339d94-b1d7-4b7d-8e61-22807a113787 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:58.193629 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5624c9-ca09-4a77-b988-5c967ef3cd97 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:13:58.206202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547574e0-9695-4b08-aa51-39d557fdb9eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:58.209264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547574e0-9695-4b08-aa51-39d557fdb9eb service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:58.209838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb14684-600b-4a76-94ca-46210cf8e8c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:13:58.211257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547574e0-9695-4b08-aa51-39d557fdb9eb service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:58.211538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547574e0-9695-4b08-aa51-39d557fdb9eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:58.212452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb14684-600b-4a76-94ca-46210cf8e8c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:13:58.212871 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb14684-600b-4a76-94ca-46210cf8e8c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:13:58.213239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb14684-600b-4a76-94ca-46210cf8e8c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:13:58.241255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547574e0-9695-4b08-aa51-39d557fdb9eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:58.243072 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547574e0-9695-4b08-aa51-39d557fdb9eb service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:58.251521 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:13:58.267911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb14684-600b-4a76-94ca-46210cf8e8c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:13:58.272513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb14684-600b-4a76-94ca-46210cf8e8c6 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:13:58.284081 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 849/1712] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:13:58.320797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 864/1713] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:13:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:14:00.388570 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547574e0-9695-4b08-aa51-39d557fdb9eb service nova] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:00.391470 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-547574e0-9695-4b08-aa51-39d557fdb9eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:00.417402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547574e0-9695-4b08-aa51-39d557fdb9eb service nova] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:00.422816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cf382-7aeb-4e3e-98a6-5543657320b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:00.423729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cf382-7aeb-4e3e-98a6-5543657320b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:00.424245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cf382-7aeb-4e3e-98a6-5543657320b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:00.424662 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cf382-7aeb-4e3e-98a6-5543657320b6 admin admin] PATH_INFO: `/v3/projects/b095553ea6ea41c38d5e712f5b93ff32` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:00.425992 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cf382-7aeb-4e3e-98a6-5543657320b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b095553ea6ea41c38d5e712f5b93ff32)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:00.428920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cf382-7aeb-4e3e-98a6-5543657320b6 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:00.753109 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb14684-600b-4a76-94ca-46210cf8e8c6 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:00.758148 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dfb14684-600b-4a76-94ca-46210cf8e8c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:00.790343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb14684-600b-4a76-94ca-46210cf8e8c6 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:00.806706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc2213-fa0e-493f-91c3-cf0363bbe677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:00.807715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc2213-fa0e-493f-91c3-cf0363bbe677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:00.808129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc2213-fa0e-493f-91c3-cf0363bbe677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:00.808625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc2213-fa0e-493f-91c3-cf0363bbe677 admin admin] PATH_INFO: `/v3/users/e82bdb1202044a638c3cd0abb19fcc31` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:00.811190 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc2213-fa0e-493f-91c3-cf0363bbe677 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e82bdb1202044a638c3cd0abb19fcc31)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:00.813439 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc2213-fa0e-493f-91c3-cf0363bbe677 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:00.921444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a28cf382-7aeb-4e3e-98a6-5543657320b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b095553ea6ea41c38d5e712f5b93ff32', 'request_id': 'req-a28cf382-7aeb-4e3e-98a6-5543657320b6'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:00.932167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a28cf382-7aeb-4e3e-98a6-5543657320b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b095553ea6ea41c38d5e712f5b93ff32', 'request_id': 'req-a28cf382-7aeb-4e3e-98a6-5543657320b6'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:01.137912 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a28cf382-7aeb-4e3e-98a6-5543657320b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b095553ea6ea41c38d5e712f5b93ff32', 'request_id': 'req-a28cf382-7aeb-4e3e-98a6-5543657320b6'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:01.192667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a28cf382-7aeb-4e3e-98a6-5543657320b6 admin admin] The token cache is being invalidate because project b095553ea6ea41c38d5e712f5b93ff32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:01.192987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a28cf382-7aeb-4e3e-98a6-5543657320b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:01.203328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a28cf382-7aeb-4e3e-98a6-5543657320b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:01.210807 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 850/1714] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:00 2022] DELETE /identity/v3/projects/b095553ea6ea41c38d5e712f5b93ff32 => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:01.220605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cf382-7aeb-4e3e-98a6-5543657320b6 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:01.223509 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a28cf382-7aeb-4e3e-98a6-5543657320b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:01.226578 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3bdc2213-fa0e-493f-91c3-cf0363bbe677 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e82bdb1202044a638c3cd0abb19fcc31'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:01.254261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:01.261883 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3bdc2213-fa0e-493f-91c3-cf0363bbe677 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e82bdb1202044a638c3cd0abb19fcc31'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:01.262291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3bdc2213-fa0e-493f-91c3-cf0363bbe677 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e82bdb1202044a638c3cd0abb19fcc31'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:01.354599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3bdc2213-fa0e-493f-91c3-cf0363bbe677 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e82bdb1202044a638c3cd0abb19fcc31'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:01.356128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3bdc2213-fa0e-493f-91c3-cf0363bbe677 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e82bdb1202044a638c3cd0abb19fcc31'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:01.383231 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 865/1715] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:00 2022] DELETE /identity/v3/users/e82bdb1202044a638c3cd0abb19fcc31 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:01.388625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e899c1-37bf-4d3d-9e60-107204f6628c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:01.389091 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e899c1-37bf-4d3d-9e60-107204f6628c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:01.389422 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e899c1-37bf-4d3d-9e60-107204f6628c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:01.416795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cf382-7aeb-4e3e-98a6-5543657320b6 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:01.436240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-83e899c1-37bf-4d3d-9e60-107204f6628c None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:01.444762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-83e899c1-37bf-4d3d-9e60-107204f6628c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:01.456271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:01.536386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5f850-301d-4144-b6ea-86a64847adce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:01.537149 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5f850-301d-4144-b6ea-86a64847adce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:01.537432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5f850-301d-4144-b6ea-86a64847adce admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:01.537758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5f850-301d-4144-b6ea-86a64847adce admin admin] PATH_INFO: `/v3/users/834b0c5265414b05bf2a5d683eee69e9` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:01.538921 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5f850-301d-4144-b6ea-86a64847adce admin admin] RBAC: Authorizing `identity:delete_user(user_id=834b0c5265414b05bf2a5d683eee69e9)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:01.541582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5f850-301d-4144-b6ea-86a64847adce admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:01.605138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 866/1716] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:01.648180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e899c1-37bf-4d3d-9e60-107204f6628c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:01.648180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-83e899c1-37bf-4d3d-9e60-107204f6628c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:01.816093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-ddc5f850-301d-4144-b6ea-86a64847adce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '834b0c5265414b05bf2a5d683eee69e9'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:01.820367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-ddc5f850-301d-4144-b6ea-86a64847adce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '834b0c5265414b05bf2a5d683eee69e9'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:01.827824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e899c1-37bf-4d3d-9e60-107204f6628c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:01.841322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417af352-2883-4bf8-bc1a-518a6ec5f86a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:01.841936 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417af352-2883-4bf8-bc1a-518a6ec5f86a service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:01.842211 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417af352-2883-4bf8-bc1a-518a6ec5f86a service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:01.842543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417af352-2883-4bf8-bc1a-518a6ec5f86a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:01.855428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-417af352-2883-4bf8-bc1a-518a6ec5f86a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:01.856606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-ddc5f850-301d-4144-b6ea-86a64847adce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '834b0c5265414b05bf2a5d683eee69e9'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:01.910521 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417af352-2883-4bf8-bc1a-518a6ec5f86a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:01.918636 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417af352-2883-4bf8-bc1a-518a6ec5f86a service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:01.960230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-ddc5f850-301d-4144-b6ea-86a64847adce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '834b0c5265414b05bf2a5d683eee69e9'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:01.960963 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-ddc5f850-301d-4144-b6ea-86a64847adce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '834b0c5265414b05bf2a5d683eee69e9'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:01.969201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 867/1717] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:14:01.982559 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 851/1718] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:01 2022] DELETE /identity/v3/users/834b0c5265414b05bf2a5d683eee69e9 => generated 0 bytes in 767 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:02.888599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb81bdd-2a00-4797-83b8-a7d158d55e0a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:02.889155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb81bdd-2a00-4797-83b8-a7d158d55e0a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:02.889155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb81bdd-2a00-4797-83b8-a7d158d55e0a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:02.891213 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 868/1719] 10.222.0.30 () {54 vars in 921 bytes} [Thu Oct 6 08:14:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:14:02.981631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abdfe2-eb60-41e9-8027-eeec02ad7a31 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:02.982096 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abdfe2-eb60-41e9-8027-eeec02ad7a31 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:02.987259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abdfe2-eb60-41e9-8027-eeec02ad7a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:03.032192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-e6abdfe2-eb60-41e9-8027-eeec02ad7a31 None None] MFA Rules not processed for user `209795285384486183cda455fe316bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:03.035859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e6abdfe2-eb60-41e9-8027-eeec02ad7a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:03.213937 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 852/1720] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Oct 6 08:14:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:03.960671 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb81bdd-2a00-4797-83b8-a7d158d55e0a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:03.963382 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1cb81bdd-2a00-4797-83b8-a7d158d55e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:04.022651 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb81bdd-2a00-4797-83b8-a7d158d55e0a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:04.035754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e91a81-f66a-40fd-9127-b334b036d1ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:04.036507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e91a81-f66a-40fd-9127-b334b036d1ef service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:04.036793 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e91a81-f66a-40fd-9127-b334b036d1ef service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:04.037117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e91a81-f66a-40fd-9127-b334b036d1ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:04.062380 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e91a81-f66a-40fd-9127-b334b036d1ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:04.064669 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e91a81-f66a-40fd-9127-b334b036d1ef service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:04.099045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 869/1721] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:14:04.987596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abdfe2-eb60-41e9-8027-eeec02ad7a31 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:04.989615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e6abdfe2-eb60-41e9-8027-eeec02ad7a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:05.098040 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:05.173038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abdfe2-eb60-41e9-8027-eeec02ad7a31 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:05.179139 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c16ba4-1436-42ae-a9c9-0dbc46c1bf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:05.179814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c16ba4-1436-42ae-a9c9-0dbc46c1bf7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:05.180858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c16ba4-1436-42ae-a9c9-0dbc46c1bf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:05.180858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c16ba4-1436-42ae-a9c9-0dbc46c1bf7d admin admin] PATH_INFO: `/v3/projects/6fb9c90ca9b14e939f3c6744555f9f4d` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:05.182526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c16ba4-1436-42ae-a9c9-0dbc46c1bf7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fb9c90ca9b14e939f3c6744555f9f4d)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:05.185341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c16ba4-1436-42ae-a9c9-0dbc46c1bf7d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:05.444716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e91a81-f66a-40fd-9127-b334b036d1ef service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:05.446364 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-50e91a81-f66a-40fd-9127-b334b036d1ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:05.464612 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e91a81-f66a-40fd-9127-b334b036d1ef service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:05.471003 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:05.471638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:05.471915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:05.472358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b admin admin] PATH_INFO: `/v3/projects/eafd10a389bb46449029240ec52d3b0c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:05.473585 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eafd10a389bb46449029240ec52d3b0c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:05.474885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 06 08:14:05.474885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Oct 06 08:14:05.474885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: Oct 06 08:14:05.476729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:05.627690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e5c16ba4-1436-42ae-a9c9-0dbc46c1bf7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fb9c90ca9b14e939f3c6744555f9f4d', 'request_id': 'req-e5c16ba4-1436-42ae-a9c9-0dbc46c1bf7d'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:05.637856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e5c16ba4-1436-42ae-a9c9-0dbc46c1bf7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fb9c90ca9b14e939f3c6744555f9f4d', 'request_id': 'req-e5c16ba4-1436-42ae-a9c9-0dbc46c1bf7d'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:05.699497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eafd10a389bb46449029240ec52d3b0c', 'request_id': 'req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:05.715695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eafd10a389bb46449029240ec52d3b0c', 'request_id': 'req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:05.749467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e5c16ba4-1436-42ae-a9c9-0dbc46c1bf7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fb9c90ca9b14e939f3c6744555f9f4d', 'request_id': 'req-e5c16ba4-1436-42ae-a9c9-0dbc46c1bf7d'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:05.763367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e5c16ba4-1436-42ae-a9c9-0dbc46c1bf7d admin admin] The token cache is being invalidate because project 6fb9c90ca9b14e939f3c6744555f9f4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:05.763696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e5c16ba4-1436-42ae-a9c9-0dbc46c1bf7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:05.764289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e5c16ba4-1436-42ae-a9c9-0dbc46c1bf7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:05.767269 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 853/1722] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:04 2022] DELETE /identity/v3/projects/6fb9c90ca9b14e939f3c6744555f9f4d => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:05.773009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617be93a-5906-4fb6-b0f3-4ef22b20b963 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:05.773519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617be93a-5906-4fb6-b0f3-4ef22b20b963 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:05.774007 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617be93a-5906-4fb6-b0f3-4ef22b20b963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:05.817001 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-617be93a-5906-4fb6-b0f3-4ef22b20b963 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:05.819342 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-617be93a-5906-4fb6-b0f3-4ef22b20b963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:05.889830 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eafd10a389bb46449029240ec52d3b0c', 'request_id': 'req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:05.890933 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:05.917384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b admin admin] The token cache is being invalidate because project eafd10a389bb46449029240ec52d3b0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:05.919083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:05.920763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:05.927390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 870/1723] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:05 2022] DELETE /identity/v3/projects/eafd10a389bb46449029240ec52d3b0c => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:05.929874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:05.934086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:05.968976 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 854/1724] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:05.979016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c667a-e631-4262-ac19-9a649aacf292 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:05.980364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c667a-e631-4262-ac19-9a649aacf292 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:05.980364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c667a-e631-4262-ac19-9a649aacf292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:05.984953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4e8e1-b1d9-4839-965b-f6f2773df63b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:05.990714 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77f08b-e293-400d-8653-35a15b0f74cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:05.991915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77f08b-e293-400d-8653-35a15b0f74cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:05.992397 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77f08b-e293-400d-8653-35a15b0f74cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:05.992813 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77f08b-e293-400d-8653-35a15b0f74cb admin admin] PATH_INFO: `/v3/users/46343636bb2c4f9e9050c4d074485a98` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:05.994376 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77f08b-e293-400d-8653-35a15b0f74cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=46343636bb2c4f9e9050c4d074485a98)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:05.996870 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77f08b-e293-400d-8653-35a15b0f74cb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:06.023032 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-f62c667a-e631-4262-ac19-9a649aacf292 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:06.025524 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f62c667a-e631-4262-ac19-9a649aacf292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:06.039183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 855/1725] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:06.043433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c667a-e631-4262-ac19-9a649aacf292 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:06.050993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c667a-e631-4262-ac19-9a649aacf292 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:06.057001 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73debd2-b7da-40d2-bccd-a3856dd5f8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:06.057718 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73debd2-b7da-40d2-bccd-a3856dd5f8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:06.058028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73debd2-b7da-40d2-bccd-a3856dd5f8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:06.058603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73debd2-b7da-40d2-bccd-a3856dd5f8d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:06.059293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73debd2-b7da-40d2-bccd-a3856dd5f8d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:14:06.059597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73debd2-b7da-40d2-bccd-a3856dd5f8d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:06.061229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73debd2-b7da-40d2-bccd-a3856dd5f8d0 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:06.078889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 856/1726] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:14:06.083131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73debd2-b7da-40d2-bccd-a3856dd5f8d0 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:06.096568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73debd2-b7da-40d2-bccd-a3856dd5f8d0 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:06.105047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f31e38-2484-4734-8e3b-bbb741d3018b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:06.105911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f31e38-2484-4734-8e3b-bbb741d3018b admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:06.106193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f31e38-2484-4734-8e3b-bbb741d3018b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:06.107157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f31e38-2484-4734-8e3b-bbb741d3018b admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:06.108155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f31e38-2484-4734-8e3b-bbb741d3018b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:06.110043 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f31e38-2484-4734-8e3b-bbb741d3018b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:06.258825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 857/1727] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:06 2022] POST /identity/v3/projects => generated 363 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:14:06.264134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f31e38-2484-4734-8e3b-bbb741d3018b admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:06.280231 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6a77f08b-e293-400d-8653-35a15b0f74cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46343636bb2c4f9e9050c4d074485a98'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:06.317859 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6a77f08b-e293-400d-8653-35a15b0f74cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46343636bb2c4f9e9050c4d074485a98'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:06.318779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6a77f08b-e293-400d-8653-35a15b0f74cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46343636bb2c4f9e9050c4d074485a98'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:06.356153 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f31e38-2484-4734-8e3b-bbb741d3018b admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:06.361440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654fdea-7912-4b0f-9352-af2aa96c40d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:06.362075 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654fdea-7912-4b0f-9352-af2aa96c40d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:06.362354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654fdea-7912-4b0f-9352-af2aa96c40d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:06.362622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654fdea-7912-4b0f-9352-af2aa96c40d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:06.363387 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654fdea-7912-4b0f-9352-af2aa96c40d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:06.365498 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654fdea-7912-4b0f-9352-af2aa96c40d9 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:06.400884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6a77f08b-e293-400d-8653-35a15b0f74cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46343636bb2c4f9e9050c4d074485a98'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:06.401545 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6a77f08b-e293-400d-8653-35a15b0f74cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46343636bb2c4f9e9050c4d074485a98'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:06.414464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 871/1728] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:05 2022] DELETE /identity/v3/users/46343636bb2c4f9e9050c4d074485a98 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:06.506067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 858/1729] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:06 2022] POST /identity/v3/users => generated 282 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:06.533822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77f08b-e293-400d-8653-35a15b0f74cb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:06.533822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6a77f08b-e293-400d-8653-35a15b0f74cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:06.582060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77f08b-e293-400d-8653-35a15b0f74cb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:06.586625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20fd1a-5ec2-4c93-b5a2-0685618029b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:06.587302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20fd1a-5ec2-4c93-b5a2-0685618029b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:06.588276 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20fd1a-5ec2-4c93-b5a2-0685618029b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:06.588563 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20fd1a-5ec2-4c93-b5a2-0685618029b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:06.589198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20fd1a-5ec2-4c93-b5a2-0685618029b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:06.590806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20fd1a-5ec2-4c93-b5a2-0685618029b2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:06.600579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 872/1730] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:14:06.605438 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654fdea-7912-4b0f-9352-af2aa96c40d9 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:06.615745 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654fdea-7912-4b0f-9352-af2aa96c40d9 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:06.620317 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f694af-03a7-40c3-a692-525ce2ae5c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:06.620933 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f694af-03a7-40c3-a692-525ce2ae5c5a admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:06.621275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f694af-03a7-40c3-a692-525ce2ae5c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:06.621509 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f694af-03a7-40c3-a692-525ce2ae5c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:06.622172 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f694af-03a7-40c3-a692-525ce2ae5c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:06.623788 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f694af-03a7-40c3-a692-525ce2ae5c5a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:06.635533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 859/1731] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:06.639713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20fd1a-5ec2-4c93-b5a2-0685618029b2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:06.650777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20fd1a-5ec2-4c93-b5a2-0685618029b2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:06.658335 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff58e3-0680-4ab4-9bce-41637f0b958f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:06.659027 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff58e3-0680-4ab4-9bce-41637f0b958f admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:06.659497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff58e3-0680-4ab4-9bce-41637f0b958f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:06.659954 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff58e3-0680-4ab4-9bce-41637f0b958f admin admin] PATH_INFO: `/v3/projects/ce61f413876b4ce7a26c8d04929f495e/users/8b0b8e8ae5794a9795c8576ffba38ecd/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:06.679092 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff58e3-0680-4ab4-9bce-41637f0b958f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce61f413876b4ce7a26c8d04929f495e, user_id=8b0b8e8ae5794a9795c8576ffba38ecd, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:06.682928 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff58e3-0680-4ab4-9bce-41637f0b958f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:06.874833 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 873/1732] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:06 2022] PUT /identity/v3/projects/ce61f413876b4ce7a26c8d04929f495e/users/8b0b8e8ae5794a9795c8576ffba38ecd/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:06.885406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a551dd8-c73d-4d69-af4d-a6f72cad3b76 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:06.885751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a551dd8-c73d-4d69-af4d-a6f72cad3b76 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:06.885961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a551dd8-c73d-4d69-af4d-a6f72cad3b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:06.927567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-4a551dd8-c73d-4d69-af4d-a6f72cad3b76 None None] MFA Rules not processed for user `8b0b8e8ae5794a9795c8576ffba38ecd`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:06.929028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4a551dd8-c73d-4d69-af4d-a6f72cad3b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:06.993128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff58e3-0680-4ab4-9bce-41637f0b958f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:06.995698 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-54ff58e3-0680-4ab4-9bce-41637f0b958f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:07.004471 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 860/1733] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:07.010224 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a551dd8-c73d-4d69-af4d-a6f72cad3b76 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:07.085931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a551dd8-c73d-4d69-af4d-a6f72cad3b76 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:07.086527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff58e3-0680-4ab4-9bce-41637f0b958f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:07.092121 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f303579-df5e-4903-8491-4cccfabc86e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:07.092762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f303579-df5e-4903-8491-4cccfabc86e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:07.093037 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f303579-df5e-4903-8491-4cccfabc86e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:07.093380 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f303579-df5e-4903-8491-4cccfabc86e8 admin admin] PATH_INFO: `/v3/projects/6e8e833656e0447081bfd873d988cb43` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:07.094439 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c6a5f-273a-46c0-b2b6-863880ce0af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:07.094570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f303579-df5e-4903-8491-4cccfabc86e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e8e833656e0447081bfd873d988cb43)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:07.094656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c6a5f-273a-46c0-b2b6-863880ce0af6 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:07.094920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c6a5f-273a-46c0-b2b6-863880ce0af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:07.095797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c6a5f-273a-46c0-b2b6-863880ce0af6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:07.107172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f303579-df5e-4903-8491-4cccfabc86e8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:07.107341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c6a5f-273a-46c0-b2b6-863880ce0af6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:07.107341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c6a5f-273a-46c0-b2b6-863880ce0af6 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:07.256352 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 861/1734] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:07 2022] POST /identity/v3/projects => generated 365 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:14:07.257420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c6a5f-273a-46c0-b2b6-863880ce0af6 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:07.277720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9f303579-df5e-4903-8491-4cccfabc86e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e8e833656e0447081bfd873d988cb43', 'request_id': 'req-9f303579-df5e-4903-8491-4cccfabc86e8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:07.284253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9f303579-df5e-4903-8491-4cccfabc86e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e8e833656e0447081bfd873d988cb43', 'request_id': 'req-9f303579-df5e-4903-8491-4cccfabc86e8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:07.338261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c6a5f-273a-46c0-b2b6-863880ce0af6 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:07.384062 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9f303579-df5e-4903-8491-4cccfabc86e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e8e833656e0447081bfd873d988cb43', 'request_id': 'req-9f303579-df5e-4903-8491-4cccfabc86e8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:07.401020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9f303579-df5e-4903-8491-4cccfabc86e8 admin admin] The token cache is being invalidate because project 6e8e833656e0447081bfd873d988cb43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:07.401414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9f303579-df5e-4903-8491-4cccfabc86e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:07.402142 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9f303579-df5e-4903-8491-4cccfabc86e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:07.403688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55514de7-7f29-44a9-aa8f-7425957cd631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:07.404052 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 874/1735] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:06 2022] DELETE /identity/v3/projects/6e8e833656e0447081bfd873d988cb43 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:07.407500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55514de7-7f29-44a9-aa8f-7425957cd631 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:07.411777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55514de7-7f29-44a9-aa8f-7425957cd631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:07.411777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55514de7-7f29-44a9-aa8f-7425957cd631 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:07.412002 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397fcf2-ca30-4489-bac5-87ee09a12c3f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:07.412002 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397fcf2-ca30-4489-bac5-87ee09a12c3f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:07.412168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55514de7-7f29-44a9-aa8f-7425957cd631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:07.412258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397fcf2-ca30-4489-bac5-87ee09a12c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:07.412356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55514de7-7f29-44a9-aa8f-7425957cd631 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:07.429406 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7397fcf2-ca30-4489-bac5-87ee09a12c3f None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:07.430732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7397fcf2-ca30-4489-bac5-87ee09a12c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:07.439371 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 875/1736] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:07.448145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c1ddd-8b20-4ea9-bfc8-023a42582b18 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:07.448463 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c1ddd-8b20-4ea9-bfc8-023a42582b18 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:07.448740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c1ddd-8b20-4ea9-bfc8-023a42582b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:07.468765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2a1c1ddd-8b20-4ea9-bfc8-023a42582b18 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:07.471454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2a1c1ddd-8b20-4ea9-bfc8-023a42582b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:07.477862 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 876/1737] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:07.482054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c1ddd-8b20-4ea9-bfc8-023a42582b18 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:07.491386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c1ddd-8b20-4ea9-bfc8-023a42582b18 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:07.496833 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04a70f-eaa0-48ec-92e4-be6fa92b1dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:07.497435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04a70f-eaa0-48ec-92e4-be6fa92b1dde admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:07.497706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04a70f-eaa0-48ec-92e4-be6fa92b1dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:07.498180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04a70f-eaa0-48ec-92e4-be6fa92b1dde admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:07.498681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04a70f-eaa0-48ec-92e4-be6fa92b1dde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:14:07.499010 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04a70f-eaa0-48ec-92e4-be6fa92b1dde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:07.500735 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04a70f-eaa0-48ec-92e4-be6fa92b1dde admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:07.500820 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 862/1738] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:07 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:14:07.501854 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55514de7-7f29-44a9-aa8f-7425957cd631 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:07.503893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-55514de7-7f29-44a9-aa8f-7425957cd631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:07.518476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 877/1739] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 08:14:07.518636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55514de7-7f29-44a9-aa8f-7425957cd631 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:07.519817 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04a70f-eaa0-48ec-92e4-be6fa92b1dde admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:07.527398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef924f1-9cd9-4a0e-b6dc-639ff1fa80c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:07.528212 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef924f1-9cd9-4a0e-b6dc-639ff1fa80c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:07.528497 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef924f1-9cd9-4a0e-b6dc-639ff1fa80c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:07.528769 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef924f1-9cd9-4a0e-b6dc-639ff1fa80c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:07.529883 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04a70f-eaa0-48ec-92e4-be6fa92b1dde admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:07.530789 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef924f1-9cd9-4a0e-b6dc-639ff1fa80c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:07.532435 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef924f1-9cd9-4a0e-b6dc-639ff1fa80c5 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:07.534516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d0452-1f63-4c19-b1f9-39840877dcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:07.540299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d0452-1f63-4c19-b1f9-39840877dcd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:07.540299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d0452-1f63-4c19-b1f9-39840877dcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:07.540299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d0452-1f63-4c19-b1f9-39840877dcd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:07.540299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d0452-1f63-4c19-b1f9-39840877dcd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:07.540299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d0452-1f63-4c19-b1f9-39840877dcd0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:07.543212 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 863/1740] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:14:07.547619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef924f1-9cd9-4a0e-b6dc-639ff1fa80c5 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:07.558227 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef924f1-9cd9-4a0e-b6dc-639ff1fa80c5 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:07.569190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e90bd8-27ca-4014-9c2f-0f84fbd32fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:07.569190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e90bd8-27ca-4014-9c2f-0f84fbd32fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:07.569190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e90bd8-27ca-4014-9c2f-0f84fbd32fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:07.569190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e90bd8-27ca-4014-9c2f-0f84fbd32fd1 admin admin] PATH_INFO: `/v3/projects/c0832afc18104b239bdb8f3c1d66753a/users/8e5a6a8cbefc4d248a86a839cfc88e84/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:07.580036 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e90bd8-27ca-4014-9c2f-0f84fbd32fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0832afc18104b239bdb8f3c1d66753a, user_id=8e5a6a8cbefc4d248a86a839cfc88e84, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:07.582812 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e90bd8-27ca-4014-9c2f-0f84fbd32fd1 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:07.630176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 878/1741] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:07 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:14:07.634361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d0452-1f63-4c19-b1f9-39840877dcd0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:07.657991 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 864/1742] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:07 2022] PUT /identity/v3/projects/c0832afc18104b239bdb8f3c1d66753a/users/8e5a6a8cbefc4d248a86a839cfc88e84/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:14:07.659176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e90bd8-27ca-4014-9c2f-0f84fbd32fd1 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:07.705291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d0452-1f63-4c19-b1f9-39840877dcd0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:07.746970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e90bd8-27ca-4014-9c2f-0f84fbd32fd1 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:07.746970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f6065-f8a2-4839-9855-6098ead93cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:07.746970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f6065-f8a2-4839-9855-6098ead93cdc admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:07.746970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f6065-f8a2-4839-9855-6098ead93cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:07.747566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f6065-f8a2-4839-9855-6098ead93cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:07.747566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f6065-f8a2-4839-9855-6098ead93cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:07.747566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f6065-f8a2-4839-9855-6098ead93cdc admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:07.754301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 865/1743] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:07.757995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f6065-f8a2-4839-9855-6098ead93cdc admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:07.768578 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f6065-f8a2-4839-9855-6098ead93cdc admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:07.769130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a4e23-5277-4919-8166-fed06c183248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:07.769732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a4e23-5277-4919-8166-fed06c183248 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:07.770002 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a4e23-5277-4919-8166-fed06c183248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:07.770272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a4e23-5277-4919-8166-fed06c183248 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:07.770926 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a4e23-5277-4919-8166-fed06c183248 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:07.772851 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a4e23-5277-4919-8166-fed06c183248 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:07.773673 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f9131-f89d-4329-a06a-1474368efbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:07.774324 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f9131-f89d-4329-a06a-1474368efbfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:07.774601 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f9131-f89d-4329-a06a-1474368efbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:07.775542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f9131-f89d-4329-a06a-1474368efbfe admin admin] PATH_INFO: `/v3/projects/c0832afc18104b239bdb8f3c1d66753a/users/8e5a6a8cbefc4d248a86a839cfc88e84/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:07.777985 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f9131-f89d-4329-a06a-1474368efbfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0832afc18104b239bdb8f3c1d66753a, user_id=8e5a6a8cbefc4d248a86a839cfc88e84, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:07.780841 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f9131-f89d-4329-a06a-1474368efbfe admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:07.791786 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 866/1744] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:07 2022] PUT /identity/v3/projects/c0832afc18104b239bdb8f3c1d66753a/users/8e5a6a8cbefc4d248a86a839cfc88e84/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:14:07.801942 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c67760-1e0b-4afc-805f-53c8f250f7ba None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:07.802224 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c67760-1e0b-4afc-805f-53c8f250f7ba None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:07.802496 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c67760-1e0b-4afc-805f-53c8f250f7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:07.830764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-79c67760-1e0b-4afc-805f-53c8f250f7ba None None] MFA Rules not processed for user `8e5a6a8cbefc4d248a86a839cfc88e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:07.832006 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-79c67760-1e0b-4afc-805f-53c8f250f7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:07.899222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 867/1745] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:07.911326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c67760-1e0b-4afc-805f-53c8f250f7ba None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:07.912742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-79c67760-1e0b-4afc-805f-53c8f250f7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:07.914655 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 879/1746] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:07 2022] POST /identity/v3/users => generated 285 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:14:07.919518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a4e23-5277-4919-8166-fed06c183248 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:07.931314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a4e23-5277-4919-8166-fed06c183248 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:07.990377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c67760-1e0b-4afc-805f-53c8f250f7ba None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:07.990377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea705d9f-fe3d-45ff-aea0-b18de8214562 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:07.990377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea705d9f-fe3d-45ff-aea0-b18de8214562 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:07.990377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea705d9f-fe3d-45ff-aea0-b18de8214562 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:07.991069 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea705d9f-fe3d-45ff-aea0-b18de8214562 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:07.994231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea705d9f-fe3d-45ff-aea0-b18de8214562 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:07.995726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea705d9f-fe3d-45ff-aea0-b18de8214562 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:08.006463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 868/1747] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:14:08.012234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dac5ff-de84-46c8-9fb0-4ad3390fe4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:08.012234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dac5ff-de84-46c8-9fb0-4ad3390fe4ad admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:08.012234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dac5ff-de84-46c8-9fb0-4ad3390fe4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:08.012234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dac5ff-de84-46c8-9fb0-4ad3390fe4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:08.012791 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dac5ff-de84-46c8-9fb0-4ad3390fe4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:08.012910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dac5ff-de84-46c8-9fb0-4ad3390fe4ad admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:08.027262 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 880/1748] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:14:08.031202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea705d9f-fe3d-45ff-aea0-b18de8214562 service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:08.041776 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea705d9f-fe3d-45ff-aea0-b18de8214562 service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:08.046441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748f991-369c-42bc-a197-5ab827fc4c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:08.047081 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748f991-369c-42bc-a197-5ab827fc4c3a admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:08.047364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748f991-369c-42bc-a197-5ab827fc4c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:08.047662 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748f991-369c-42bc-a197-5ab827fc4c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:08.048332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748f991-369c-42bc-a197-5ab827fc4c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:08.049972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748f991-369c-42bc-a197-5ab827fc4c3a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:08.061972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 869/1749] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:08.065816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dac5ff-de84-46c8-9fb0-4ad3390fe4ad admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:08.076215 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dac5ff-de84-46c8-9fb0-4ad3390fe4ad admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:08.080951 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76128b-5188-486e-9558-53104f1c23bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:08.081600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76128b-5188-486e-9558-53104f1c23bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:08.081873 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76128b-5188-486e-9558-53104f1c23bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:08.082141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76128b-5188-486e-9558-53104f1c23bf admin admin] PATH_INFO: `/v3/projects/64efdb6f1fb24200b7ef51dc6b3d01b2/users/325e5fe669ca46f2b44b7427728df000/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:08.097306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76128b-5188-486e-9558-53104f1c23bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=64efdb6f1fb24200b7ef51dc6b3d01b2, user_id=325e5fe669ca46f2b44b7427728df000, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:08.100137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76128b-5188-486e-9558-53104f1c23bf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:08.177780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:08.184113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 881/1750] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:08 2022] PUT /identity/v3/projects/64efdb6f1fb24200b7ef51dc6b3d01b2/users/325e5fe669ca46f2b44b7427728df000/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:08.201861 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f8ebf-1119-4c3b-a3e7-486cb9b95614 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:08.202156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f8ebf-1119-4c3b-a3e7-486cb9b95614 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:08.202513 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f8ebf-1119-4c3b-a3e7-486cb9b95614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:08.235483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-040f8ebf-1119-4c3b-a3e7-486cb9b95614 None None] MFA Rules not processed for user `325e5fe669ca46f2b44b7427728df000`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:08.236759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-040f8ebf-1119-4c3b-a3e7-486cb9b95614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:08.301747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 870/1751] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:08.307116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76128b-5188-486e-9558-53104f1c23bf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:08.383401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:08.387730 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76128b-5188-486e-9558-53104f1c23bf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:08.392498 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f13aa4-3ffe-4dac-8f9c-a5358d3553b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:08.393725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f13aa4-3ffe-4dac-8f9c-a5358d3553b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:08.393725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f13aa4-3ffe-4dac-8f9c-a5358d3553b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:08.393725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f13aa4-3ffe-4dac-8f9c-a5358d3553b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:08.394224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f13aa4-3ffe-4dac-8f9c-a5358d3553b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:08.396152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f13aa4-3ffe-4dac-8f9c-a5358d3553b1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:08.475902 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 882/1752] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:08 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:14:08.481339 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f8ebf-1119-4c3b-a3e7-486cb9b95614 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:08.496193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f8ebf-1119-4c3b-a3e7-486cb9b95614 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:08.558027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b0856-db61-4daf-a1c9-4c385693f816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:08.558640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b0856-db61-4daf-a1c9-4c385693f816 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:08.562007 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b0856-db61-4daf-a1c9-4c385693f816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:08.562007 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b0856-db61-4daf-a1c9-4c385693f816 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:08.562007 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b0856-db61-4daf-a1c9-4c385693f816 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:08.562007 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b0856-db61-4daf-a1c9-4c385693f816 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:08.648050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 871/1753] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:08 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:14:08.649128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f13aa4-3ffe-4dac-8f9c-a5358d3553b1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:08.660748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f13aa4-3ffe-4dac-8f9c-a5358d3553b1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:08.667456 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c295b7b-d100-4ed3-b699-a24aa5ac28fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:08.668276 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c295b7b-d100-4ed3-b699-a24aa5ac28fb admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:08.668478 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c295b7b-d100-4ed3-b699-a24aa5ac28fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:08.668731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c295b7b-d100-4ed3-b699-a24aa5ac28fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:08.669375 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c295b7b-d100-4ed3-b699-a24aa5ac28fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:08.670995 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c295b7b-d100-4ed3-b699-a24aa5ac28fb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:08.682919 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 883/1754] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:14:08.687587 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b0856-db61-4daf-a1c9-4c385693f816 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:08.701940 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b0856-db61-4daf-a1c9-4c385693f816 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:08.707440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95484ce1-626d-4098-99ea-1fc0bcc1d79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:08.708406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95484ce1-626d-4098-99ea-1fc0bcc1d79a admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:08.708931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95484ce1-626d-4098-99ea-1fc0bcc1d79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:08.709275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95484ce1-626d-4098-99ea-1fc0bcc1d79a admin admin] PATH_INFO: `/v3/projects/f58ae89a2b32405fb27111ac51db7380/users/77530319589a43598a374bcee335c2dd/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:08.734911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95484ce1-626d-4098-99ea-1fc0bcc1d79a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f58ae89a2b32405fb27111ac51db7380, user_id=77530319589a43598a374bcee335c2dd, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:08.737746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95484ce1-626d-4098-99ea-1fc0bcc1d79a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:08.869480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 872/1755] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:08 2022] PUT /identity/v3/projects/f58ae89a2b32405fb27111ac51db7380/users/77530319589a43598a374bcee335c2dd/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:08.873453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c295b7b-d100-4ed3-b699-a24aa5ac28fb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:08.883570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c295b7b-d100-4ed3-b699-a24aa5ac28fb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:08.941195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a54ed2-28b8-4e39-9258-816efaf5b7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:08.941803 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a54ed2-28b8-4e39-9258-816efaf5b7ce admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:08.942082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a54ed2-28b8-4e39-9258-816efaf5b7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:08.942355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a54ed2-28b8-4e39-9258-816efaf5b7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:08.943030 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a54ed2-28b8-4e39-9258-816efaf5b7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:08.944643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a54ed2-28b8-4e39-9258-816efaf5b7ce admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:08.954025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 884/1756] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:08.958253 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95484ce1-626d-4098-99ea-1fc0bcc1d79a admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:08.969550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95484ce1-626d-4098-99ea-1fc0bcc1d79a admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:08.974227 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb3504-9767-421f-b0f9-560e45b1e038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:08.974903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb3504-9767-421f-b0f9-560e45b1e038 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:08.975227 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb3504-9767-421f-b0f9-560e45b1e038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:08.975463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb3504-9767-421f-b0f9-560e45b1e038 admin admin] PATH_INFO: `/v3/projects/f58ae89a2b32405fb27111ac51db7380/users/77530319589a43598a374bcee335c2dd/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:08.977379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb3504-9767-421f-b0f9-560e45b1e038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f58ae89a2b32405fb27111ac51db7380, user_id=77530319589a43598a374bcee335c2dd, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:08.980675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb3504-9767-421f-b0f9-560e45b1e038 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:08.990933 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 873/1757] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:08 2022] PUT /identity/v3/projects/f58ae89a2b32405fb27111ac51db7380/users/77530319589a43598a374bcee335c2dd/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:09.001184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1d0c7-78a2-47a7-a857-b3d656957de6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:09.001467 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1d0c7-78a2-47a7-a857-b3d656957de6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:09.003785 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1d0c7-78a2-47a7-a857-b3d656957de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:09.031273 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-26e1d0c7-78a2-47a7-a857-b3d656957de6 None None] MFA Rules not processed for user `77530319589a43598a374bcee335c2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:09.032491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-26e1d0c7-78a2-47a7-a857-b3d656957de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:09.063030 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb3504-9767-421f-b0f9-560e45b1e038 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:09.102191 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 885/1758] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:09.114280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1d0c7-78a2-47a7-a857-b3d656957de6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:09.115659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-26e1d0c7-78a2-47a7-a857-b3d656957de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:09.126414 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb3504-9767-421f-b0f9-560e45b1e038 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:09.131588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a0af0-721b-489c-86b3-4ba476b40864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:09.132250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a0af0-721b-489c-86b3-4ba476b40864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:09.132525 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a0af0-721b-489c-86b3-4ba476b40864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:09.132790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a0af0-721b-489c-86b3-4ba476b40864 admin admin] PATH_INFO: `/v3/users/8b0b8e8ae5794a9795c8576ffba38ecd` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:09.133937 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a0af0-721b-489c-86b3-4ba476b40864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b0b8e8ae5794a9795c8576ffba38ecd)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:09.136645 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a0af0-721b-489c-86b3-4ba476b40864 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:09.178089 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1d0c7-78a2-47a7-a857-b3d656957de6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:09.182025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cca49-2e68-418f-b2ff-496beb721ce3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:09.182625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cca49-2e68-418f-b2ff-496beb721ce3 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:09.182898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cca49-2e68-418f-b2ff-496beb721ce3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:09.183198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cca49-2e68-418f-b2ff-496beb721ce3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:09.193072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5cca49-2e68-418f-b2ff-496beb721ce3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:09.194622 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5cca49-2e68-418f-b2ff-496beb721ce3 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:09.206042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 886/1759] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:14:09.346827 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-627a0af0-721b-489c-86b3-4ba476b40864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b0b8e8ae5794a9795c8576ffba38ecd'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:09.347773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-627a0af0-721b-489c-86b3-4ba476b40864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b0b8e8ae5794a9795c8576ffba38ecd'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:09.360221 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-627a0af0-721b-489c-86b3-4ba476b40864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b0b8e8ae5794a9795c8576ffba38ecd'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:09.428327 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-627a0af0-721b-489c-86b3-4ba476b40864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b0b8e8ae5794a9795c8576ffba38ecd'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:09.428975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-627a0af0-721b-489c-86b3-4ba476b40864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b0b8e8ae5794a9795c8576ffba38ecd'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:09.443064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 874/1760] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:09 2022] DELETE /identity/v3/users/8b0b8e8ae5794a9795c8576ffba38ecd => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:14:09.448178 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2273dd-0f44-46c7-91a9-af5aa13524e1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:09.449779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2273dd-0f44-46c7-91a9-af5aa13524e1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:09.449779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2273dd-0f44-46c7-91a9-af5aa13524e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:09.469641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bb2273dd-0f44-46c7-91a9-af5aa13524e1 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:09.470890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bb2273dd-0f44-46c7-91a9-af5aa13524e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:09.541200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 887/1761] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:09.562038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a0af0-721b-489c-86b3-4ba476b40864 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:09.562038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-627a0af0-721b-489c-86b3-4ba476b40864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:09.625062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a0af0-721b-489c-86b3-4ba476b40864 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:09.629171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459a5b7-e266-4862-87f0-b5a0043e2bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:09.629845 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459a5b7-e266-4862-87f0-b5a0043e2bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:09.630196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459a5b7-e266-4862-87f0-b5a0043e2bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:09.630537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459a5b7-e266-4862-87f0-b5a0043e2bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:09.640691 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459a5b7-e266-4862-87f0-b5a0043e2bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:09.642244 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459a5b7-e266-4862-87f0-b5a0043e2bf8 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:09.652336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 875/1762] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:14:09.733048 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:10.092435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:10.240800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2273dd-0f44-46c7-91a9-af5aa13524e1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:10.242756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bb2273dd-0f44-46c7-91a9-af5aa13524e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:10.255852 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2273dd-0f44-46c7-91a9-af5aa13524e1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:10.260707 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:10.267154 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:10.267154 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:10.267154 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3 admin admin] PATH_INFO: `/v3/projects/ce61f413876b4ce7a26c8d04929f495e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:10.267154 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce61f413876b4ce7a26c8d04929f495e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:10.267154 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:10.319486 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459a5b7-e266-4862-87f0-b5a0043e2bf8 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:10.321122 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4459a5b7-e266-4862-87f0-b5a0043e2bf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:10.334032 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459a5b7-e266-4862-87f0-b5a0043e2bf8 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:10.340487 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a06c1-c629-4dd4-93c2-d3662fb77a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:10.341125 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a06c1-c629-4dd4-93c2-d3662fb77a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:10.341409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a06c1-c629-4dd4-93c2-d3662fb77a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:10.341672 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a06c1-c629-4dd4-93c2-d3662fb77a2b admin admin] PATH_INFO: `/v3/users/325e5fe669ca46f2b44b7427728df000` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:10.342768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a06c1-c629-4dd4-93c2-d3662fb77a2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=325e5fe669ca46f2b44b7427728df000)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:10.344865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a06c1-c629-4dd4-93c2-d3662fb77a2b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:10.424818 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce61f413876b4ce7a26c8d04929f495e', 'request_id': 'req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:10.438048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce61f413876b4ce7a26c8d04929f495e', 'request_id': 'req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:10.526911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce61f413876b4ce7a26c8d04929f495e', 'request_id': 'req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:10.539157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-df4a06c1-c629-4dd4-93c2-d3662fb77a2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '325e5fe669ca46f2b44b7427728df000'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:10.541050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-df4a06c1-c629-4dd4-93c2-d3662fb77a2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '325e5fe669ca46f2b44b7427728df000'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:10.544580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3 admin admin] The token cache is being invalidate because project ce61f413876b4ce7a26c8d04929f495e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:10.545291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:10.546009 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:10.548510 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 888/1763] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:10 2022] DELETE /identity/v3/projects/ce61f413876b4ce7a26c8d04929f495e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:10.552974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:10.555329 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:10.562166 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-df4a06c1-c629-4dd4-93c2-d3662fb77a2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '325e5fe669ca46f2b44b7427728df000'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:10.693847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f3c6c-f041-4673-a36f-fc4c6c3783b3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:10.700487 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc1d73-f527-48be-96d6-faa105cc77b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:10.702415 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc1d73-f527-48be-96d6-faa105cc77b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:10.702415 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc1d73-f527-48be-96d6-faa105cc77b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:10.702415 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc1d73-f527-48be-96d6-faa105cc77b3 admin admin] PATH_INFO: `/v3/users/8e5a6a8cbefc4d248a86a839cfc88e84` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:10.703448 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc1d73-f527-48be-96d6-faa105cc77b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e5a6a8cbefc4d248a86a839cfc88e84)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:10.716527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc1d73-f527-48be-96d6-faa105cc77b3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:11.331404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-df4a06c1-c629-4dd4-93c2-d3662fb77a2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '325e5fe669ca46f2b44b7427728df000'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:11.332170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-df4a06c1-c629-4dd4-93c2-d3662fb77a2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '325e5fe669ca46f2b44b7427728df000'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:11.346064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 876/1764] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:10 2022] DELETE /identity/v3/users/325e5fe669ca46f2b44b7427728df000 => generated 0 bytes in 1027 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:11.350490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea04b02-5444-499a-90c9-845774140bfe None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:11.350720 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea04b02-5444-499a-90c9-845774140bfe None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:11.351038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea04b02-5444-499a-90c9-845774140bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:11.371918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-9ea04b02-5444-499a-90c9-845774140bfe None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:11.373369 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9ea04b02-5444-499a-90c9-845774140bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:11.434027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 877/1765] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:11.443772 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea04b02-5444-499a-90c9-845774140bfe None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:11.445120 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9ea04b02-5444-499a-90c9-845774140bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:11.504090 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea04b02-5444-499a-90c9-845774140bfe None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:11.507797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313bfd9-a1bc-43bc-a362-c62b4d4a861f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:11.508421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313bfd9-a1bc-43bc-a362-c62b4d4a861f service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:11.508725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313bfd9-a1bc-43bc-a362-c62b4d4a861f service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:11.509512 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313bfd9-a1bc-43bc-a362-c62b4d4a861f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:11.521187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7313bfd9-a1bc-43bc-a362-c62b4d4a861f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:11.522721 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7313bfd9-a1bc-43bc-a362-c62b4d4a861f service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:11.534073 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 878/1766] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:14:12.108195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-56dc1d73-f527-48be-96d6-faa105cc77b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e5a6a8cbefc4d248a86a839cfc88e84'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:12.122910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-56dc1d73-f527-48be-96d6-faa105cc77b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e5a6a8cbefc4d248a86a839cfc88e84'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:12.124394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-56dc1d73-f527-48be-96d6-faa105cc77b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e5a6a8cbefc4d248a86a839cfc88e84'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:12.216811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-56dc1d73-f527-48be-96d6-faa105cc77b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e5a6a8cbefc4d248a86a839cfc88e84'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:12.217480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-56dc1d73-f527-48be-96d6-faa105cc77b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e5a6a8cbefc4d248a86a839cfc88e84'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:12.237216 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 889/1767] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:10 2022] DELETE /identity/v3/users/8e5a6a8cbefc4d248a86a839cfc88e84 => generated 0 bytes in 1685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:12.320031 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:12.814258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313bfd9-a1bc-43bc-a362-c62b4d4a861f service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:12.815845 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7313bfd9-a1bc-43bc-a362-c62b4d4a861f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:12.830433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313bfd9-a1bc-43bc-a362-c62b4d4a861f service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:12.882455 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc1d73-f527-48be-96d6-faa105cc77b3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:12.884215 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-56dc1d73-f527-48be-96d6-faa105cc77b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:12.894490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d70bf-3668-4fac-9037-5e3d6c4167c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:12.898078 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d70bf-3668-4fac-9037-5e3d6c4167c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:12.898078 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d70bf-3668-4fac-9037-5e3d6c4167c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:12.898078 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d70bf-3668-4fac-9037-5e3d6c4167c6 admin admin] PATH_INFO: `/v3/projects/64efdb6f1fb24200b7ef51dc6b3d01b2` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:12.898078 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d70bf-3668-4fac-9037-5e3d6c4167c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64efdb6f1fb24200b7ef51dc6b3d01b2)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:12.901333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d70bf-3668-4fac-9037-5e3d6c4167c6 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:12.956792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc1d73-f527-48be-96d6-faa105cc77b3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:12.961985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f0aa5-78e2-4859-9f76-23fbdc18cef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:12.962734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f0aa5-78e2-4859-9f76-23fbdc18cef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:12.963031 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f0aa5-78e2-4859-9f76-23fbdc18cef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:12.963308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f0aa5-78e2-4859-9f76-23fbdc18cef4 admin admin] PATH_INFO: `/v3/projects/c0832afc18104b239bdb8f3c1d66753a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:12.964441 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f0aa5-78e2-4859-9f76-23fbdc18cef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0832afc18104b239bdb8f3c1d66753a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:12.966370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f0aa5-78e2-4859-9f76-23fbdc18cef4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:13.022121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d77d70bf-3668-4fac-9037-5e3d6c4167c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64efdb6f1fb24200b7ef51dc6b3d01b2', 'request_id': 'req-d77d70bf-3668-4fac-9037-5e3d6c4167c6'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:13.028248 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d77d70bf-3668-4fac-9037-5e3d6c4167c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64efdb6f1fb24200b7ef51dc6b3d01b2', 'request_id': 'req-d77d70bf-3668-4fac-9037-5e3d6c4167c6'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:13.114590 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ae8f0aa5-78e2-4859-9f76-23fbdc18cef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0832afc18104b239bdb8f3c1d66753a', 'request_id': 'req-ae8f0aa5-78e2-4859-9f76-23fbdc18cef4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:13.119139 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d77d70bf-3668-4fac-9037-5e3d6c4167c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64efdb6f1fb24200b7ef51dc6b3d01b2', 'request_id': 'req-d77d70bf-3668-4fac-9037-5e3d6c4167c6'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:13.122578 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ae8f0aa5-78e2-4859-9f76-23fbdc18cef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0832afc18104b239bdb8f3c1d66753a', 'request_id': 'req-ae8f0aa5-78e2-4859-9f76-23fbdc18cef4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:13.131013 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d77d70bf-3668-4fac-9037-5e3d6c4167c6 admin admin] The token cache is being invalidate because project 64efdb6f1fb24200b7ef51dc6b3d01b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:13.131305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d77d70bf-3668-4fac-9037-5e3d6c4167c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:13.131977 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d77d70bf-3668-4fac-9037-5e3d6c4167c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:13.133487 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 879/1768] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:12 2022] DELETE /identity/v3/projects/64efdb6f1fb24200b7ef51dc6b3d01b2 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:13.137817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d70bf-3668-4fac-9037-5e3d6c4167c6 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:13.139148 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d77d70bf-3668-4fac-9037-5e3d6c4167c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:13.194227 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:13.203791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d70bf-3668-4fac-9037-5e3d6c4167c6 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:13.228307 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ae8f0aa5-78e2-4859-9f76-23fbdc18cef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0832afc18104b239bdb8f3c1d66753a', 'request_id': 'req-ae8f0aa5-78e2-4859-9f76-23fbdc18cef4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:13.241132 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ae8f0aa5-78e2-4859-9f76-23fbdc18cef4 admin admin] The token cache is being invalidate because project c0832afc18104b239bdb8f3c1d66753a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:13.241132 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ae8f0aa5-78e2-4859-9f76-23fbdc18cef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:13.241958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ae8f0aa5-78e2-4859-9f76-23fbdc18cef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:13.244585 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 890/1769] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:12 2022] DELETE /identity/v3/projects/c0832afc18104b239bdb8f3c1d66753a => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:14:13.249226 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb9bd3-de35-4f88-a96b-c21329cdfdc0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:13.249589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb9bd3-de35-4f88-a96b-c21329cdfdc0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:13.250168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb9bd3-de35-4f88-a96b-c21329cdfdc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:13.267341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55da01-3099-403e-9243-7535a55d59c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:13.268077 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55da01-3099-403e-9243-7535a55d59c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:13.268427 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55da01-3099-403e-9243-7535a55d59c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:13.268760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55da01-3099-403e-9243-7535a55d59c9 admin admin] PATH_INFO: `/v3/users/77530319589a43598a374bcee335c2dd` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:13.270981 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55da01-3099-403e-9243-7535a55d59c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77530319589a43598a374bcee335c2dd)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:13.273144 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55da01-3099-403e-9243-7535a55d59c9 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:13.278970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1ddb9bd3-de35-4f88-a96b-c21329cdfdc0 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:13.278970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1ddb9bd3-de35-4f88-a96b-c21329cdfdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:13.286899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 891/1770] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:13.296530 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186b429-df91-4b07-8e4d-040e4b01ff66 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:13.296808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186b429-df91-4b07-8e4d-040e4b01ff66 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:13.297258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186b429-df91-4b07-8e4d-040e4b01ff66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:13.319107 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6186b429-df91-4b07-8e4d-040e4b01ff66 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:13.320427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6186b429-df91-4b07-8e4d-040e4b01ff66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:13.328852 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 892/1771] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:13.332843 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186b429-df91-4b07-8e4d-040e4b01ff66 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:13.342735 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186b429-df91-4b07-8e4d-040e4b01ff66 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:13.347656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdee50a-9338-464e-921d-06a7600ecf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:13.348247 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdee50a-9338-464e-921d-06a7600ecf37 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:13.348525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdee50a-9338-464e-921d-06a7600ecf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:13.348795 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdee50a-9338-464e-921d-06a7600ecf37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:13.349451 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdee50a-9338-464e-921d-06a7600ecf37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:14:13.349753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdee50a-9338-464e-921d-06a7600ecf37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:13.351473 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdee50a-9338-464e-921d-06a7600ecf37 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:13.364615 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 893/1772] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:14:13.379080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdee50a-9338-464e-921d-06a7600ecf37 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:13.379390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdee50a-9338-464e-921d-06a7600ecf37 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:13.384116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b205fc-4245-4284-bde5-bd51415e1842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:13.384754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b205fc-4245-4284-bde5-bd51415e1842 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:13.385028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b205fc-4245-4284-bde5-bd51415e1842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:13.385300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b205fc-4245-4284-bde5-bd51415e1842 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:13.385930 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b205fc-4245-4284-bde5-bd51415e1842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:13.387876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b205fc-4245-4284-bde5-bd51415e1842 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:13.435339 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-4a55da01-3099-403e-9243-7535a55d59c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77530319589a43598a374bcee335c2dd'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:13.435997 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-4a55da01-3099-403e-9243-7535a55d59c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77530319589a43598a374bcee335c2dd'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:13.447933 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-4a55da01-3099-403e-9243-7535a55d59c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77530319589a43598a374bcee335c2dd'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:13.506483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 894/1773] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:13 2022] POST /identity/v3/projects => generated 355 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:14:13.510647 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b205fc-4245-4284-bde5-bd51415e1842 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:13.512765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d0b205fc-4245-4284-bde5-bd51415e1842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:13.524432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-4a55da01-3099-403e-9243-7535a55d59c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77530319589a43598a374bcee335c2dd'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:13.525811 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-4a55da01-3099-403e-9243-7535a55d59c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77530319589a43598a374bcee335c2dd'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:13.550489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 880/1774] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:13 2022] DELETE /identity/v3/users/77530319589a43598a374bcee335c2dd => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:14:13.593184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b205fc-4245-4284-bde5-bd51415e1842 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:13.611377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:13.658723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92b508-3bbb-4711-af55-b0a6b3fc4fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:13.659393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92b508-3bbb-4711-af55-b0a6b3fc4fdd admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:13.659738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92b508-3bbb-4711-af55-b0a6b3fc4fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:13.659941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92b508-3bbb-4711-af55-b0a6b3fc4fdd admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:13.660879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92b508-3bbb-4711-af55-b0a6b3fc4fdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:13.662654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92b508-3bbb-4711-af55-b0a6b3fc4fdd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:13.753848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 895/1775] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:13 2022] POST /identity/v3/users => generated 278 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:13.754156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55da01-3099-403e-9243-7535a55d59c9 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:13.764441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55da01-3099-403e-9243-7535a55d59c9 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:13.769163 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b636c0-3494-422d-9f3e-939adf86023e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:13.771960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b636c0-3494-422d-9f3e-939adf86023e admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:13.772278 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b636c0-3494-422d-9f3e-939adf86023e admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:13.772565 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b636c0-3494-422d-9f3e-939adf86023e admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:13.773741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b636c0-3494-422d-9f3e-939adf86023e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:13.776005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b636c0-3494-422d-9f3e-939adf86023e admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:13.787802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 881/1776] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:13.792553 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92b508-3bbb-4711-af55-b0a6b3fc4fdd admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:13.802831 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92b508-3bbb-4711-af55-b0a6b3fc4fdd admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:13.811049 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7cd41a-e79e-478a-830f-67565d1babbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:13.811049 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cd41a-e79e-478a-830f-67565d1babbf admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:13.811408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cd41a-e79e-478a-830f-67565d1babbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:13.814199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cd41a-e79e-478a-830f-67565d1babbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:13.814199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7cd41a-e79e-478a-830f-67565d1babbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:13.814199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7cd41a-e79e-478a-830f-67565d1babbf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:13.825275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 896/1777] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:14:13.829707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b636c0-3494-422d-9f3e-939adf86023e admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:13.840591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b636c0-3494-422d-9f3e-939adf86023e admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:13.846132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ac89b-2d8b-4c26-a648-62dfc4be3418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:13.846835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ac89b-2d8b-4c26-a648-62dfc4be3418 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:13.847152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ac89b-2d8b-4c26-a648-62dfc4be3418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:13.847427 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ac89b-2d8b-4c26-a648-62dfc4be3418 admin admin] PATH_INFO: `/v3/projects/08dbba82e2b74248a34b366190b1199e/users/2cd9d816179d407e819a69809837a595/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:13.867086 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ac89b-2d8b-4c26-a648-62dfc4be3418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08dbba82e2b74248a34b366190b1199e, user_id=2cd9d816179d407e819a69809837a595, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:13.870200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ac89b-2d8b-4c26-a648-62dfc4be3418 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:13.949273 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 882/1778] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:13 2022] PUT /identity/v3/projects/08dbba82e2b74248a34b366190b1199e/users/2cd9d816179d407e819a69809837a595/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:13.960736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f0fb9-679a-455f-a1aa-2c0ef60c2874 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:13.961018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f0fb9-679a-455f-a1aa-2c0ef60c2874 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:13.961292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f0fb9-679a-455f-a1aa-2c0ef60c2874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:13.989959 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cf3f0fb9-679a-455f-a1aa-2c0ef60c2874 None None] MFA Rules not processed for user `2cd9d816179d407e819a69809837a595`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:13.991147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cf3f0fb9-679a-455f-a1aa-2c0ef60c2874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:14.063545 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 897/1779] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:13 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:14.068186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ac89b-2d8b-4c26-a648-62dfc4be3418 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:14.130134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ac89b-2d8b-4c26-a648-62dfc4be3418 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:14.135538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f101ae-cd5c-4cb9-a6df-b608a5ea56da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:14.136135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f101ae-cd5c-4cb9-a6df-b608a5ea56da admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:14.136410 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f101ae-cd5c-4cb9-a6df-b608a5ea56da admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:14.136674 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f101ae-cd5c-4cb9-a6df-b608a5ea56da admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:14.137295 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f101ae-cd5c-4cb9-a6df-b608a5ea56da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:14.139265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f101ae-cd5c-4cb9-a6df-b608a5ea56da admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:14.166516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f0fb9-679a-455f-a1aa-2c0ef60c2874 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:14.168163 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cf3f0fb9-679a-455f-a1aa-2c0ef60c2874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:14.182970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f0fb9-679a-455f-a1aa-2c0ef60c2874 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:14.187808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb49e5-05f4-4558-b275-da81025d050d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:14.188920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb49e5-05f4-4558-b275-da81025d050d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:14.189366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb49e5-05f4-4558-b275-da81025d050d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:14.189638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb49e5-05f4-4558-b275-da81025d050d admin admin] PATH_INFO: `/v3/projects/f58ae89a2b32405fb27111ac51db7380` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:14.190813 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb49e5-05f4-4558-b275-da81025d050d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f58ae89a2b32405fb27111ac51db7380)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:14.192815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb49e5-05f4-4558-b275-da81025d050d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:14.245377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 883/1780] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:14 2022] POST /identity/v3/projects => generated 357 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:14.247298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f101ae-cd5c-4cb9-a6df-b608a5ea56da admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:14.257302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:14.293517 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d4cb49e5-05f4-4558-b275-da81025d050d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f58ae89a2b32405fb27111ac51db7380', 'request_id': 'req-d4cb49e5-05f4-4558-b275-da81025d050d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:14.299823 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d4cb49e5-05f4-4558-b275-da81025d050d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f58ae89a2b32405fb27111ac51db7380', 'request_id': 'req-d4cb49e5-05f4-4558-b275-da81025d050d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:14.311628 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f101ae-cd5c-4cb9-a6df-b608a5ea56da admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:14.366864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc585c3a-f2e2-41d4-be8c-f1d8512555b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:14.367514 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc585c3a-f2e2-41d4-be8c-f1d8512555b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:14.367796 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc585c3a-f2e2-41d4-be8c-f1d8512555b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:14.368073 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc585c3a-f2e2-41d4-be8c-f1d8512555b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:14.370690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc585c3a-f2e2-41d4-be8c-f1d8512555b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:14.370889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d4cb49e5-05f4-4558-b275-da81025d050d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f58ae89a2b32405fb27111ac51db7380', 'request_id': 'req-d4cb49e5-05f4-4558-b275-da81025d050d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:14.371450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc585c3a-f2e2-41d4-be8c-f1d8512555b2 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:14.383447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d4cb49e5-05f4-4558-b275-da81025d050d admin admin] The token cache is being invalidate because project f58ae89a2b32405fb27111ac51db7380 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:14.383754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d4cb49e5-05f4-4558-b275-da81025d050d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:14.385189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d4cb49e5-05f4-4558-b275-da81025d050d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:14.390455 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 898/1781] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:14 2022] DELETE /identity/v3/projects/f58ae89a2b32405fb27111ac51db7380 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:14:14.391842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4af8bb-0c19-41a2-ab71-253ba327ac80 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:14.392119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4af8bb-0c19-41a2-ab71-253ba327ac80 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:14.392427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4af8bb-0c19-41a2-ab71-253ba327ac80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:14.414525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bf4af8bb-0c19-41a2-ab71-253ba327ac80 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:14.416402 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bf4af8bb-0c19-41a2-ab71-253ba327ac80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:14.425105 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 899/1782] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:14.435011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026effb-1ccf-4a76-91cd-1dd32ed5320e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:14.438931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026effb-1ccf-4a76-91cd-1dd32ed5320e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:14.438931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026effb-1ccf-4a76-91cd-1dd32ed5320e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:14.442910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 884/1783] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:14 2022] POST /identity/v3/users => generated 285 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:14.449227 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc585c3a-f2e2-41d4-be8c-f1d8512555b2 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:14.451184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-dc585c3a-f2e2-41d4-be8c-f1d8512555b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:14.464129 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc585c3a-f2e2-41d4-be8c-f1d8512555b2 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:14.469565 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317cdd2-fe5b-4a2d-996a-c7f32b290c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:14.470020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d026effb-1ccf-4a76-91cd-1dd32ed5320e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:14.470183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317cdd2-fe5b-4a2d-996a-c7f32b290c8c admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:14.470460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317cdd2-fe5b-4a2d-996a-c7f32b290c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:14.470730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317cdd2-fe5b-4a2d-996a-c7f32b290c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:14.471424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317cdd2-fe5b-4a2d-996a-c7f32b290c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:14.471770 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d026effb-1ccf-4a76-91cd-1dd32ed5320e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:14.473069 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317cdd2-fe5b-4a2d-996a-c7f32b290c8c admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:14.486982 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 900/1784] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:14.487200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 885/1785] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:14:14.488227 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026effb-1ccf-4a76-91cd-1dd32ed5320e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:14.490544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317cdd2-fe5b-4a2d-996a-c7f32b290c8c admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:14.502180 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317cdd2-fe5b-4a2d-996a-c7f32b290c8c admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:14.507935 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c59c23-dbb2-48c7-8db3-e5e4b0971c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:14.508977 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c59c23-dbb2-48c7-8db3-e5e4b0971c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:14.509459 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c59c23-dbb2-48c7-8db3-e5e4b0971c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:14.509951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c59c23-dbb2-48c7-8db3-e5e4b0971c06 admin admin] PATH_INFO: `/v3/projects/82be3d00897a480ca14060d4ced5f0be/users/e20105e54e6f4d9b830929a95007a1ec/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:14.515087 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:14.529895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026effb-1ccf-4a76-91cd-1dd32ed5320e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:14.542451 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c59c23-dbb2-48c7-8db3-e5e4b0971c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82be3d00897a480ca14060d4ced5f0be, user_id=e20105e54e6f4d9b830929a95007a1ec, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:14.544914 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a70f97-4734-41f1-a662-947e1fb80b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:14.545362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a70f97-4734-41f1-a662-947e1fb80b86 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:14.547295 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a70f97-4734-41f1-a662-947e1fb80b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:14.547742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a70f97-4734-41f1-a662-947e1fb80b86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:14.548125 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c59c23-dbb2-48c7-8db3-e5e4b0971c06 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:14.550309 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a70f97-4734-41f1-a662-947e1fb80b86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:14:14.550309 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a70f97-4734-41f1-a662-947e1fb80b86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:14.552224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a70f97-4734-41f1-a662-947e1fb80b86 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:14.572457 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 901/1786] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:14:14.586390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a70f97-4734-41f1-a662-947e1fb80b86 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:14.587847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a70f97-4734-41f1-a662-947e1fb80b86 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:14.594231 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bbd432-e3a7-447c-be9c-f7801c194777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:14.595444 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbd432-e3a7-447c-be9c-f7801c194777 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:14.595960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbd432-e3a7-447c-be9c-f7801c194777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:14.596427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbd432-e3a7-447c-be9c-f7801c194777 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:14.597263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bbd432-e3a7-447c-be9c-f7801c194777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:14.599750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bbd432-e3a7-447c-be9c-f7801c194777 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:14.640587 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:14.647320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 886/1787] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:14 2022] PUT /identity/v3/projects/82be3d00897a480ca14060d4ced5f0be/users/e20105e54e6f4d9b830929a95007a1ec/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:14:14.651739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c59c23-dbb2-48c7-8db3-e5e4b0971c06 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:14.749798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 902/1788] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:14 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:14.754107 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bbd432-e3a7-447c-be9c-f7801c194777 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:14.802202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c59c23-dbb2-48c7-8db3-e5e4b0971c06 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:14.865227 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bbd432-e3a7-447c-be9c-f7801c194777 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:14.872668 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d6177-7762-45b5-b4f4-e4aea98d77a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:14.873547 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d6177-7762-45b5-b4f4-e4aea98d77a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:14.874006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d6177-7762-45b5-b4f4-e4aea98d77a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:14.874430 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d6177-7762-45b5-b4f4-e4aea98d77a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:14.875755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d6177-7762-45b5-b4f4-e4aea98d77a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:14.879352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d6177-7762-45b5-b4f4-e4aea98d77a9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:14.887217 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d3cd7-6836-49ed-990c-37e5a29260e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:14.887217 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d3cd7-6836-49ed-990c-37e5a29260e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:14.887217 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d3cd7-6836-49ed-990c-37e5a29260e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:14.887217 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d3cd7-6836-49ed-990c-37e5a29260e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:14.887812 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d3cd7-6836-49ed-990c-37e5a29260e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:14.887812 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d3cd7-6836-49ed-990c-37e5a29260e9 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:14.903026 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 887/1789] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:14:14.904422 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d3cd7-6836-49ed-990c-37e5a29260e9 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:14.916549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d3cd7-6836-49ed-990c-37e5a29260e9 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:14.921489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b7143-ab42-499c-af19-bc40028cf6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:14.922339 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b7143-ab42-499c-af19-bc40028cf6e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:14.922705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b7143-ab42-499c-af19-bc40028cf6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:14.927320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b7143-ab42-499c-af19-bc40028cf6e3 admin admin] PATH_INFO: `/v3/projects/82be3d00897a480ca14060d4ced5f0be/users/e20105e54e6f4d9b830929a95007a1ec/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:14.929798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b7143-ab42-499c-af19-bc40028cf6e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82be3d00897a480ca14060d4ced5f0be, user_id=e20105e54e6f4d9b830929a95007a1ec, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:14.933142 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b7143-ab42-499c-af19-bc40028cf6e3 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:14.963081 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 888/1790] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:14 2022] PUT /identity/v3/projects/82be3d00897a480ca14060d4ced5f0be/users/e20105e54e6f4d9b830929a95007a1ec/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:14:14.965044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7359e3-3bb8-49b4-ab9e-93f60685f6f1 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:14.965044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7359e3-3bb8-49b4-ab9e-93f60685f6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:14.965356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7359e3-3bb8-49b4-ab9e-93f60685f6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:15.011064 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 903/1791] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:14 2022] POST /identity/v3/users => generated 294 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:15.012348 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d6177-7762-45b5-b4f4-e4aea98d77a9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:15.039261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-dc7359e3-3bb8-49b4-ab9e-93f60685f6f1 None None] MFA Rules not processed for user `e20105e54e6f4d9b830929a95007a1ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:15.040908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-dc7359e3-3bb8-49b4-ab9e-93f60685f6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:15.042373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d6177-7762-45b5-b4f4-e4aea98d77a9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:15.139887 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 889/1792] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:14 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:15.205714 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c01099-731e-42a8-be9b-08ca70c186f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:15.205714 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c01099-731e-42a8-be9b-08ca70c186f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:15.205714 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c01099-731e-42a8-be9b-08ca70c186f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:15.205714 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c01099-731e-42a8-be9b-08ca70c186f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:15.206282 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c01099-731e-42a8-be9b-08ca70c186f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:15.206282 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c01099-731e-42a8-be9b-08ca70c186f9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:15.206449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7359e3-3bb8-49b4-ab9e-93f60685f6f1 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:15.206449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-dc7359e3-3bb8-49b4-ab9e-93f60685f6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:15.206608 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 904/1793] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:14:15.207999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c01099-731e-42a8-be9b-08ca70c186f9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:15.221419 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c01099-731e-42a8-be9b-08ca70c186f9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:15.228089 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ff713-6d30-4570-be4b-9b453a59ce75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:15.229134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ff713-6d30-4570-be4b-9b453a59ce75 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:15.229592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ff713-6d30-4570-be4b-9b453a59ce75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:15.229863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ff713-6d30-4570-be4b-9b453a59ce75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:15.230918 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ff713-6d30-4570-be4b-9b453a59ce75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:15.233102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ff713-6d30-4570-be4b-9b453a59ce75 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:15.246730 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 905/1794] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:15.251380 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ff713-6d30-4570-be4b-9b453a59ce75 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:15.254132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7359e3-3bb8-49b4-ab9e-93f60685f6f1 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:15.258395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a5da8-3e50-4f99-ac83-4f4d046279f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:15.259547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a5da8-3e50-4f99-ac83-4f4d046279f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:15.259904 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a5da8-3e50-4f99-ac83-4f4d046279f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:15.260202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a5da8-3e50-4f99-ac83-4f4d046279f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:15.272678 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a5da8-3e50-4f99-ac83-4f4d046279f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:15.274267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a5da8-3e50-4f99-ac83-4f4d046279f5 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:15.276183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ff713-6d30-4570-be4b-9b453a59ce75 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:15.283648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c58b5-dd79-40ae-9db4-5b179fb1c79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:15.285187 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c58b5-dd79-40ae-9db4-5b179fb1c79a admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:15.286295 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c58b5-dd79-40ae-9db4-5b179fb1c79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:15.286750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c58b5-dd79-40ae-9db4-5b179fb1c79a admin admin] PATH_INFO: `/v3/projects/3f0ce2d2124046e68dccd80455e66cd8/users/673ec18bda31457bb16683fd6c480c7c/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:15.296445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 890/1795] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:15 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:14:15.310382 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c58b5-dd79-40ae-9db4-5b179fb1c79a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f0ce2d2124046e68dccd80455e66cd8, user_id=673ec18bda31457bb16683fd6c480c7c, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:15.313108 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c58b5-dd79-40ae-9db4-5b179fb1c79a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:16.006773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 906/1796] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:15 2022] PUT /identity/v3/projects/3f0ce2d2124046e68dccd80455e66cd8/users/673ec18bda31457bb16683fd6c480c7c/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:14:16.018346 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c3c41-f763-4ca2-9afc-27b945b581c0 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:16.018874 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c3c41-f763-4ca2-9afc-27b945b581c0 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:16.018874 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c3c41-f763-4ca2-9afc-27b945b581c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:16.051837 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-0f2c3c41-f763-4ca2-9afc-27b945b581c0 None None] MFA Rules not processed for user `673ec18bda31457bb16683fd6c480c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:16.057428 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0f2c3c41-f763-4ca2-9afc-27b945b581c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:16.120088 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 891/1797] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:16.124784 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c58b5-dd79-40ae-9db4-5b179fb1c79a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:16.164946 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c3c41-f763-4ca2-9afc-27b945b581c0 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:16.200961 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c58b5-dd79-40ae-9db4-5b179fb1c79a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:16.206232 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bfd5d-ec11-4073-8df3-81cc60abf06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:16.207015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bfd5d-ec11-4073-8df3-81cc60abf06f admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:16.207291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bfd5d-ec11-4073-8df3-81cc60abf06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:16.207560 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bfd5d-ec11-4073-8df3-81cc60abf06f admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:16.208370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bfd5d-ec11-4073-8df3-81cc60abf06f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:16.210293 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bfd5d-ec11-4073-8df3-81cc60abf06f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:16.234119 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c3c41-f763-4ca2-9afc-27b945b581c0 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:16.237910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38894c-e4cd-48de-b2d7-c5afdcdfb497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:16.242863 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38894c-e4cd-48de-b2d7-c5afdcdfb497 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:16.242863 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38894c-e4cd-48de-b2d7-c5afdcdfb497 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:16.242863 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38894c-e4cd-48de-b2d7-c5afdcdfb497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:16.242863 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-bd38894c-e4cd-48de-b2d7-c5afdcdfb497 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:16.302275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 907/1798] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:16 2022] POST /identity/v3/projects => generated 387 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:16.303401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bfd5d-ec11-4073-8df3-81cc60abf06f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:16.310126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38894c-e4cd-48de-b2d7-c5afdcdfb497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:16.311663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38894c-e4cd-48de-b2d7-c5afdcdfb497 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:16.323622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:16.327546 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 892/1799] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:14:16.373576 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:16.377832 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bfd5d-ec11-4073-8df3-81cc60abf06f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:16.383457 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a7215-3099-4887-bbe7-a925554a287e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:16.384695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a7215-3099-4887-bbe7-a925554a287e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:16.384942 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a7215-3099-4887-bbe7-a925554a287e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:16.385660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a7215-3099-4887-bbe7-a925554a287e admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:16.385987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a7215-3099-4887-bbe7-a925554a287e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:16.387994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a7215-3099-4887-bbe7-a925554a287e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:16.501758 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 908/1800] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:16 2022] POST /identity/v3/users => generated 300 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:16.505936 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38894c-e4cd-48de-b2d7-c5afdcdfb497 service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:16.519533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38894c-e4cd-48de-b2d7-c5afdcdfb497 service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:16.526524 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68b275-3028-4572-b3ad-2a17620dedf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:16.527159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68b275-3028-4572-b3ad-2a17620dedf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:16.535538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68b275-3028-4572-b3ad-2a17620dedf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:16.535538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68b275-3028-4572-b3ad-2a17620dedf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:16.535538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68b275-3028-4572-b3ad-2a17620dedf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:16.535538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68b275-3028-4572-b3ad-2a17620dedf5 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:16.540106 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 893/1801] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:16.544401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a7215-3099-4887-bbe7-a925554a287e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:16.556899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a7215-3099-4887-bbe7-a925554a287e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:16.563991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f46d1-a74f-4c6d-9ea3-04050cf8e1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:16.564837 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f46d1-a74f-4c6d-9ea3-04050cf8e1ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:16.565214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f46d1-a74f-4c6d-9ea3-04050cf8e1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:16.574377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f46d1-a74f-4c6d-9ea3-04050cf8e1ee admin admin] PATH_INFO: `/v3/projects/979a8c598d9a4be89ef176ba4f4a12e5/users/a792505c836f47ebb5558fd82cb6889d/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:16.583205 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f46d1-a74f-4c6d-9ea3-04050cf8e1ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=979a8c598d9a4be89ef176ba4f4a12e5, user_id=a792505c836f47ebb5558fd82cb6889d, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:16.585928 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f46d1-a74f-4c6d-9ea3-04050cf8e1ee admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:16.697025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 909/1802] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:16 2022] PUT /identity/v3/projects/979a8c598d9a4be89ef176ba4f4a12e5/users/a792505c836f47ebb5558fd82cb6889d/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:14:16.699812 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68b275-3028-4572-b3ad-2a17620dedf5 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:16.710728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68b275-3028-4572-b3ad-2a17620dedf5 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:16.770236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5053273-4f9f-43f4-83e1-d4d2f5c113dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:16.771168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5053273-4f9f-43f4-83e1-d4d2f5c113dc admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:16.771450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5053273-4f9f-43f4-83e1-d4d2f5c113dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:16.771727 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5053273-4f9f-43f4-83e1-d4d2f5c113dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:16.772374 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5053273-4f9f-43f4-83e1-d4d2f5c113dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:16.773995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5053273-4f9f-43f4-83e1-d4d2f5c113dc admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:16.789588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 894/1803] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:16.796805 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f46d1-a74f-4c6d-9ea3-04050cf8e1ee admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:16.803761 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f46d1-a74f-4c6d-9ea3-04050cf8e1ee admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:16.810048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b4fea-c693-427f-8bd5-f01164bbf1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:16.810709 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b4fea-c693-427f-8bd5-f01164bbf1ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:16.811024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b4fea-c693-427f-8bd5-f01164bbf1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:16.811313 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b4fea-c693-427f-8bd5-f01164bbf1ce admin admin] PATH_INFO: `/v3/projects/979a8c598d9a4be89ef176ba4f4a12e5/users/a792505c836f47ebb5558fd82cb6889d/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:16.813245 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b4fea-c693-427f-8bd5-f01164bbf1ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=979a8c598d9a4be89ef176ba4f4a12e5, user_id=a792505c836f47ebb5558fd82cb6889d, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:16.815949 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b4fea-c693-427f-8bd5-f01164bbf1ce admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:16.827204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 910/1804] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:16 2022] PUT /identity/v3/projects/979a8c598d9a4be89ef176ba4f4a12e5/users/a792505c836f47ebb5558fd82cb6889d/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:16.837474 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c086f-9d9d-457b-bb36-15b696235c66 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:16.837753 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c086f-9d9d-457b-bb36-15b696235c66 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:16.838019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c086f-9d9d-457b-bb36-15b696235c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:16.866243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-125c086f-9d9d-457b-bb36-15b696235c66 None None] MFA Rules not processed for user `a792505c836f47ebb5558fd82cb6889d`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:16.867472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-125c086f-9d9d-457b-bb36-15b696235c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:16.938542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 895/1805] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:16.950171 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b4fea-c693-427f-8bd5-f01164bbf1ce admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:17.018150 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b4fea-c693-427f-8bd5-f01164bbf1ce admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:17.024363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fff38a-fc63-4879-9984-0f2f73ad9fbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:17.025620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fff38a-fc63-4879-9984-0f2f73ad9fbb service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:17.025905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fff38a-fc63-4879-9984-0f2f73ad9fbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:17.026179 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fff38a-fc63-4879-9984-0f2f73ad9fbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:17.039879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fff38a-fc63-4879-9984-0f2f73ad9fbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:17.042524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fff38a-fc63-4879-9984-0f2f73ad9fbb service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:17.053994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 911/1806] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:16 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:14:17.114622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c086f-9d9d-457b-bb36-15b696235c66 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:17.190517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c086f-9d9d-457b-bb36-15b696235c66 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:17.197452 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:17.200178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da36874-ada1-4270-8c26-9a2445994659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:17.201019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da36874-ada1-4270-8c26-9a2445994659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:17.201411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da36874-ada1-4270-8c26-9a2445994659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:17.201757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da36874-ada1-4270-8c26-9a2445994659 admin admin] PATH_INFO: `/v3/users/673ec18bda31457bb16683fd6c480c7c` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:17.203114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da36874-ada1-4270-8c26-9a2445994659 admin admin] RBAC: Authorizing `identity:delete_user(user_id=673ec18bda31457bb16683fd6c480c7c)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:17.206267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da36874-ada1-4270-8c26-9a2445994659 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:17.251609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:17.391247 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-8da36874-ada1-4270-8c26-9a2445994659 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '673ec18bda31457bb16683fd6c480c7c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:17.391923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-8da36874-ada1-4270-8c26-9a2445994659 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '673ec18bda31457bb16683fd6c480c7c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:17.405468 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-8da36874-ada1-4270-8c26-9a2445994659 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '673ec18bda31457bb16683fd6c480c7c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:17.457070 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:17.485275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-8da36874-ada1-4270-8c26-9a2445994659 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '673ec18bda31457bb16683fd6c480c7c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:17.485913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-8da36874-ada1-4270-8c26-9a2445994659 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '673ec18bda31457bb16683fd6c480c7c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:17.497558 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:17.502006 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 896/1807] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:17 2022] DELETE /identity/v3/users/673ec18bda31457bb16683fd6c480c7c => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:17.507040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085f674-7781-4046-8742-bb3747ab5d25 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:17.507432 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085f674-7781-4046-8742-bb3747ab5d25 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:17.507772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085f674-7781-4046-8742-bb3747ab5d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:17.528216 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0085f674-7781-4046-8742-bb3747ab5d25 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:17.529511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0085f674-7781-4046-8742-bb3747ab5d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:17.597797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 912/1808] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:17.605290 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da36874-ada1-4270-8c26-9a2445994659 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:17.606874 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-8da36874-ada1-4270-8c26-9a2445994659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:17.668670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da36874-ada1-4270-8c26-9a2445994659 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:17.672476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b359b-5030-4123-9b8a-ede9326d0a7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:17.673089 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b359b-5030-4123-9b8a-ede9326d0a7e service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:17.673386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b359b-5030-4123-9b8a-ede9326d0a7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:17.673655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b359b-5030-4123-9b8a-ede9326d0a7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:17.684041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b359b-5030-4123-9b8a-ede9326d0a7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:17.685859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b359b-5030-4123-9b8a-ede9326d0a7e service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:17.701640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 897/1809] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:14:17.754507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:17.846331 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:18.299567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085f674-7781-4046-8742-bb3747ab5d25 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:18.302686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0085f674-7781-4046-8742-bb3747ab5d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:18.324329 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085f674-7781-4046-8742-bb3747ab5d25 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:18.329589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e430b-d914-45af-ae75-bc0ad3caa7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:18.330233 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e430b-d914-45af-ae75-bc0ad3caa7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:18.330512 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e430b-d914-45af-ae75-bc0ad3caa7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:18.330786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e430b-d914-45af-ae75-bc0ad3caa7bd admin admin] PATH_INFO: `/v3/projects/3f0ce2d2124046e68dccd80455e66cd8` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:18.332281 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e430b-d914-45af-ae75-bc0ad3caa7bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f0ce2d2124046e68dccd80455e66cd8)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:18.334373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e430b-d914-45af-ae75-bc0ad3caa7bd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:18.337771 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:18.525908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-874e430b-d914-45af-ae75-bc0ad3caa7bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f0ce2d2124046e68dccd80455e66cd8', 'request_id': 'req-874e430b-d914-45af-ae75-bc0ad3caa7bd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:18.539502 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-874e430b-d914-45af-ae75-bc0ad3caa7bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f0ce2d2124046e68dccd80455e66cd8', 'request_id': 'req-874e430b-d914-45af-ae75-bc0ad3caa7bd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:18.645563 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-874e430b-d914-45af-ae75-bc0ad3caa7bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f0ce2d2124046e68dccd80455e66cd8', 'request_id': 'req-874e430b-d914-45af-ae75-bc0ad3caa7bd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:18.659943 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-874e430b-d914-45af-ae75-bc0ad3caa7bd admin admin] The token cache is being invalidate because project 3f0ce2d2124046e68dccd80455e66cd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:18.660227 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-874e430b-d914-45af-ae75-bc0ad3caa7bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:18.661299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-874e430b-d914-45af-ae75-bc0ad3caa7bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:18.662800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 913/1810] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:18 2022] DELETE /identity/v3/projects/3f0ce2d2124046e68dccd80455e66cd8 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:18.667051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b359b-5030-4123-9b8a-ede9326d0a7e service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:18.677510 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b359b-5030-4123-9b8a-ede9326d0a7e service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:18.739419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e588518-451b-41dc-8ff5-9c01e1aba142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:18.740047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e588518-451b-41dc-8ff5-9c01e1aba142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:18.740326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e588518-451b-41dc-8ff5-9c01e1aba142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:18.740592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e588518-451b-41dc-8ff5-9c01e1aba142 admin admin] PATH_INFO: `/v3/users/a792505c836f47ebb5558fd82cb6889d` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:18.741902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e588518-451b-41dc-8ff5-9c01e1aba142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a792505c836f47ebb5558fd82cb6889d)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:18.745763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e588518-451b-41dc-8ff5-9c01e1aba142 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:18.972643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-0e588518-451b-41dc-8ff5-9c01e1aba142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a792505c836f47ebb5558fd82cb6889d'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:18.973313 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-0e588518-451b-41dc-8ff5-9c01e1aba142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a792505c836f47ebb5558fd82cb6889d'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:18.985758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-0e588518-451b-41dc-8ff5-9c01e1aba142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a792505c836f47ebb5558fd82cb6889d'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:19.068860 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:19.102051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-0e588518-451b-41dc-8ff5-9c01e1aba142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a792505c836f47ebb5558fd82cb6889d'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:19.103367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-0e588518-451b-41dc-8ff5-9c01e1aba142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a792505c836f47ebb5558fd82cb6889d'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:19.115810 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 898/1811] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:18 2022] DELETE /identity/v3/users/a792505c836f47ebb5558fd82cb6889d => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:19.298606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:19.701637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:19.710653 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e430b-d914-45af-ae75-bc0ad3caa7bd admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:19.715255 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-874e430b-d914-45af-ae75-bc0ad3caa7bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:19.736872 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e430b-d914-45af-ae75-bc0ad3caa7bd admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:19.770404 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:19.804654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61b048-7b00-4e72-8706-378f2f0d3406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:19.806491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61b048-7b00-4e72-8706-378f2f0d3406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:19.806768 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61b048-7b00-4e72-8706-378f2f0d3406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:19.807070 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61b048-7b00-4e72-8706-378f2f0d3406 admin admin] PATH_INFO: `/v3/projects/979a8c598d9a4be89ef176ba4f4a12e5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:19.808238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61b048-7b00-4e72-8706-378f2f0d3406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=979a8c598d9a4be89ef176ba4f4a12e5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:19.810277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61b048-7b00-4e72-8706-378f2f0d3406 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:19.998638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5e61b048-7b00-4e72-8706-378f2f0d3406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '979a8c598d9a4be89ef176ba4f4a12e5', 'request_id': 'req-5e61b048-7b00-4e72-8706-378f2f0d3406'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:20.006012 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5e61b048-7b00-4e72-8706-378f2f0d3406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '979a8c598d9a4be89ef176ba4f4a12e5', 'request_id': 'req-5e61b048-7b00-4e72-8706-378f2f0d3406'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:20.125425 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5e61b048-7b00-4e72-8706-378f2f0d3406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '979a8c598d9a4be89ef176ba4f4a12e5', 'request_id': 'req-5e61b048-7b00-4e72-8706-378f2f0d3406'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:20.137102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5e61b048-7b00-4e72-8706-378f2f0d3406 admin admin] The token cache is being invalidate because project 979a8c598d9a4be89ef176ba4f4a12e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:20.137391 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5e61b048-7b00-4e72-8706-378f2f0d3406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:20.138181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5e61b048-7b00-4e72-8706-378f2f0d3406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:20.140244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 914/1812] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:19 2022] DELETE /identity/v3/projects/979a8c598d9a4be89ef176ba4f4a12e5 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:20.144843 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7b83d-6e71-4394-9f59-88f4f3d14a55 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:20.145130 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7b83d-6e71-4394-9f59-88f4f3d14a55 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:20.145409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7b83d-6e71-4394-9f59-88f4f3d14a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:20.173416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-36d7b83d-6e71-4394-9f59-88f4f3d14a55 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:20.174659 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-36d7b83d-6e71-4394-9f59-88f4f3d14a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:20.248100 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 899/1813] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:20.257188 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fad06-e4fd-4d95-9e87-39f1e1657f1f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:20.257757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fad06-e4fd-4d95-9e87-39f1e1657f1f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:20.257867 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fad06-e4fd-4d95-9e87-39f1e1657f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:20.280782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-619fad06-e4fd-4d95-9e87-39f1e1657f1f None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:20.281217 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-619fad06-e4fd-4d95-9e87-39f1e1657f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:20.289374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 915/1814] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:20.294404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7b83d-6e71-4394-9f59-88f4f3d14a55 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:20.307588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7b83d-6e71-4394-9f59-88f4f3d14a55 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:20.312856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c385d-cca8-4bb1-b5f7-6ad5dec7c9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:20.313528 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c385d-cca8-4bb1-b5f7-6ad5dec7c9c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:20.313811 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c385d-cca8-4bb1-b5f7-6ad5dec7c9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:20.314253 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c385d-cca8-4bb1-b5f7-6ad5dec7c9c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:20.314913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c385d-cca8-4bb1-b5f7-6ad5dec7c9c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:14:20.315293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c385d-cca8-4bb1-b5f7-6ad5dec7c9c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:20.316938 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c385d-cca8-4bb1-b5f7-6ad5dec7c9c3 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:20.335023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 900/1815] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:14:20.336936 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619fad06-e4fd-4d95-9e87-39f1e1657f1f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:20.346761 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619fad06-e4fd-4d95-9e87-39f1e1657f1f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:20.351458 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce2fbb-f5d1-482a-825a-df63c136df66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:20.352073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce2fbb-f5d1-482a-825a-df63c136df66 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:20.352378 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce2fbb-f5d1-482a-825a-df63c136df66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:20.353044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce2fbb-f5d1-482a-825a-df63c136df66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:20.353394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce2fbb-f5d1-482a-825a-df63c136df66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:20.355393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce2fbb-f5d1-482a-825a-df63c136df66 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:20.505246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 916/1816] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:20 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:20.509309 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c385d-cca8-4bb1-b5f7-6ad5dec7c9c3 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:20.519511 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c385d-cca8-4bb1-b5f7-6ad5dec7c9c3 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:20.579410 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fd9db-a171-48a3-9957-11bbb53c1f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:20.580178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fd9db-a171-48a3-9957-11bbb53c1f83 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:20.580464 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fd9db-a171-48a3-9957-11bbb53c1f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:20.580736 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fd9db-a171-48a3-9957-11bbb53c1f83 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:20.581564 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fd9db-a171-48a3-9957-11bbb53c1f83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:20.583544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fd9db-a171-48a3-9957-11bbb53c1f83 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:20.632144 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985026a0-c0a7-45da-9812-4f3064bcaf11 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:20.632419 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985026a0-c0a7-45da-9812-4f3064bcaf11 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:20.632689 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985026a0-c0a7-45da-9812-4f3064bcaf11 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:20.634236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 917/1817] 10.222.0.30 () {54 vars in 921 bytes} [Thu Oct 6 08:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:14:20.641075 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed171209-46d9-47b8-92fb-da87c7d3af90 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:20.641570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed171209-46d9-47b8-92fb-da87c7d3af90 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:20.641840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed171209-46d9-47b8-92fb-da87c7d3af90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:20.661926 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ed171209-46d9-47b8-92fb-da87c7d3af90 None None] MFA Rules not processed for user `209795285384486183cda455fe316bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:20.663170 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ed171209-46d9-47b8-92fb-da87c7d3af90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:20.683827 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 901/1818] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:20 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:20.684888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fd9db-a171-48a3-9957-11bbb53c1f83 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:20.695134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fd9db-a171-48a3-9957-11bbb53c1f83 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:20.702516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c8e34-389c-4215-86a0-20b28c33d690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:20.703149 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c8e34-389c-4215-86a0-20b28c33d690 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:20.703428 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c8e34-389c-4215-86a0-20b28c33d690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:20.703746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c8e34-389c-4215-86a0-20b28c33d690 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:20.705749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c8e34-389c-4215-86a0-20b28c33d690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:20.707614 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c8e34-389c-4215-86a0-20b28c33d690 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:20.719407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 902/1819] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:20.723051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c8e34-389c-4215-86a0-20b28c33d690 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:20.735724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c8e34-389c-4215-86a0-20b28c33d690 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:20.741571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5ffb5-d636-4549-8029-d4fde4259a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:20.742176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5ffb5-d636-4549-8029-d4fde4259a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:20.742448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5ffb5-d636-4549-8029-d4fde4259a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:20.742801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5ffb5-d636-4549-8029-d4fde4259a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:20.744194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5ffb5-d636-4549-8029-d4fde4259a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:20.745788 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5ffb5-d636-4549-8029-d4fde4259a78 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:20.756706 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 903/1820] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:20.760695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5ffb5-d636-4549-8029-d4fde4259a78 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:20.770705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 918/1821] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Oct 6 08:14:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:20.774635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5ffb5-d636-4549-8029-d4fde4259a78 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:20.780550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fa85c-9b16-4fc7-a64d-978faf617b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:20.781251 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fa85c-9b16-4fc7-a64d-978faf617b5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:20.781533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fa85c-9b16-4fc7-a64d-978faf617b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:20.781813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fa85c-9b16-4fc7-a64d-978faf617b5f admin admin] PATH_INFO: `/v3/projects/57781ed5ea154c51b4641282fbf0f444/users/55717b0e43fb4618b92a668a36c3bb63/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:20.788927 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed171209-46d9-47b8-92fb-da87c7d3af90 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:20.791194 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ed171209-46d9-47b8-92fb-da87c7d3af90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:20.798657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fa85c-9b16-4fc7-a64d-978faf617b5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=57781ed5ea154c51b4641282fbf0f444, user_id=55717b0e43fb4618b92a668a36c3bb63, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:20.801703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fa85c-9b16-4fc7-a64d-978faf617b5f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:20.809614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed171209-46d9-47b8-92fb-da87c7d3af90 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:20.815726 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4529f-edff-4e40-bfcf-67916db306fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:20.816332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4529f-edff-4e40-bfcf-67916db306fa service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:20.816606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4529f-edff-4e40-bfcf-67916db306fa service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:20.816871 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4529f-edff-4e40-bfcf-67916db306fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:20.826427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4529f-edff-4e40-bfcf-67916db306fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:20.828283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4529f-edff-4e40-bfcf-67916db306fa service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:20.839559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 919/1822] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:14:20.886565 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 904/1823] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:20 2022] PUT /identity/v3/projects/57781ed5ea154c51b4641282fbf0f444/users/55717b0e43fb4618b92a668a36c3bb63/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:20.897398 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9361d8c-c10b-4dec-873f-ead9fdda55db None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:20.898544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9361d8c-c10b-4dec-873f-ead9fdda55db None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:20.898824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9361d8c-c10b-4dec-873f-ead9fdda55db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:20.931286 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c9361d8c-c10b-4dec-873f-ead9fdda55db None None] MFA Rules not processed for user `55717b0e43fb4618b92a668a36c3bb63`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:20.932592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c9361d8c-c10b-4dec-873f-ead9fdda55db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:20.995442 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:21.006152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 920/1824] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:21.011113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fa85c-9b16-4fc7-a64d-978faf617b5f admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:21.057691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:21.076249 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fa85c-9b16-4fc7-a64d-978faf617b5f admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:21.082725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c19147-8e0f-487c-af46-4ec6596c65d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:21.083609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c19147-8e0f-487c-af46-4ec6596c65d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:21.090028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c19147-8e0f-487c-af46-4ec6596c65d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:21.090028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c19147-8e0f-487c-af46-4ec6596c65d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:21.090028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c19147-8e0f-487c-af46-4ec6596c65d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:21.090028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c19147-8e0f-487c-af46-4ec6596c65d1 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:21.252832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 905/1825] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:21 2022] POST /identity/v3/projects => generated 385 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:14:21.260270 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9361d8c-c10b-4dec-873f-ead9fdda55db None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:21.273666 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9361d8c-c10b-4dec-873f-ead9fdda55db None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:21.339869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad98d13-87da-417e-88dd-cb7077046a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:21.340531 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad98d13-87da-417e-88dd-cb7077046a8d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:21.340810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad98d13-87da-417e-88dd-cb7077046a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:21.341077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad98d13-87da-417e-88dd-cb7077046a8d admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:21.341717 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad98d13-87da-417e-88dd-cb7077046a8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:21.343713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad98d13-87da-417e-88dd-cb7077046a8d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:21.517413 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 921/1826] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:21 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:21.519913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c19147-8e0f-487c-af46-4ec6596c65d1 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:21.531099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c19147-8e0f-487c-af46-4ec6596c65d1 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:21.536094 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7674ff6-fc14-4142-a7df-108c7ea4170e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:21.536681 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7674ff6-fc14-4142-a7df-108c7ea4170e admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:21.536961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7674ff6-fc14-4142-a7df-108c7ea4170e admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:21.537236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7674ff6-fc14-4142-a7df-108c7ea4170e admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:21.537912 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7674ff6-fc14-4142-a7df-108c7ea4170e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:21.539611 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7674ff6-fc14-4142-a7df-108c7ea4170e admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:21.550839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 906/1827] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:21.560151 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad98d13-87da-417e-88dd-cb7077046a8d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:21.570298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad98d13-87da-417e-88dd-cb7077046a8d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:21.575616 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d5e26-8675-4b1b-89cb-d8792cb6d4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:21.576218 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d5e26-8675-4b1b-89cb-d8792cb6d4a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:21.576529 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d5e26-8675-4b1b-89cb-d8792cb6d4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:21.576828 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d5e26-8675-4b1b-89cb-d8792cb6d4a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:21.577465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d5e26-8675-4b1b-89cb-d8792cb6d4a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:21.579083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d5e26-8675-4b1b-89cb-d8792cb6d4a3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:21.590835 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 922/1828] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:21.594572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7674ff6-fc14-4142-a7df-108c7ea4170e admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:21.605560 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7674ff6-fc14-4142-a7df-108c7ea4170e admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:21.610082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d0e62-80ad-4b48-9ce0-2ceb6d5b5650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:21.610732 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d0e62-80ad-4b48-9ce0-2ceb6d5b5650 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:21.611032 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d0e62-80ad-4b48-9ce0-2ceb6d5b5650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:21.611306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d0e62-80ad-4b48-9ce0-2ceb6d5b5650 admin admin] PATH_INFO: `/v3/projects/41b63c8853a94fe78e935a3badcbf26e/users/8b4871d1281c46a89faaece68e21551c/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:21.626166 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d0e62-80ad-4b48-9ce0-2ceb6d5b5650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41b63c8853a94fe78e935a3badcbf26e, user_id=8b4871d1281c46a89faaece68e21551c, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:21.629160 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d0e62-80ad-4b48-9ce0-2ceb6d5b5650 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:21.712239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 907/1829] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:21 2022] PUT /identity/v3/projects/41b63c8853a94fe78e935a3badcbf26e/users/8b4871d1281c46a89faaece68e21551c/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:21.719885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cafbe-3be2-455b-b9a5-e31236b7df67 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:21.720257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cafbe-3be2-455b-b9a5-e31236b7df67 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:21.720653 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cafbe-3be2-455b-b9a5-e31236b7df67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:21.750740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-be6cafbe-3be2-455b-b9a5-e31236b7df67 None None] MFA Rules not processed for user `8b4871d1281c46a89faaece68e21551c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:21.752075 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-be6cafbe-3be2-455b-b9a5-e31236b7df67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:21.823468 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 923/1830] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:21.827737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d0e62-80ad-4b48-9ce0-2ceb6d5b5650 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:21.896846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d0e62-80ad-4b48-9ce0-2ceb6d5b5650 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:21.905493 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17fc45-a2ec-43fd-a7a4-e291ff56ff28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:21.906218 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17fc45-a2ec-43fd-a7a4-e291ff56ff28 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:21.906471 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17fc45-a2ec-43fd-a7a4-e291ff56ff28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:21.906792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17fc45-a2ec-43fd-a7a4-e291ff56ff28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:21.907477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17fc45-a2ec-43fd-a7a4-e291ff56ff28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:21.909390 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17fc45-a2ec-43fd-a7a4-e291ff56ff28 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:21.921691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:22.004928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 908/1831] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:21 2022] POST /identity/v3/projects => generated 387 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:22.008904 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6cafbe-3be2-455b-b9a5-e31236b7df67 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:22.019677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6cafbe-3be2-455b-b9a5-e31236b7df67 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:22.082296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed52efc-c33b-480b-8875-9f473f6ed97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:22.082965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed52efc-c33b-480b-8875-9f473f6ed97e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:22.083252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed52efc-c33b-480b-8875-9f473f6ed97e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:22.083521 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed52efc-c33b-480b-8875-9f473f6ed97e admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:22.084157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed52efc-c33b-480b-8875-9f473f6ed97e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:22.087938 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed52efc-c33b-480b-8875-9f473f6ed97e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:22.252631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 924/1832] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:22 2022] POST /identity/v3/users => generated 300 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:22.254167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17fc45-a2ec-43fd-a7a4-e291ff56ff28 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:22.272622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17fc45-a2ec-43fd-a7a4-e291ff56ff28 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:22.279823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256fa2f-9d49-4ce4-87eb-70c49cc454be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:22.280536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256fa2f-9d49-4ce4-87eb-70c49cc454be admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:22.280816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256fa2f-9d49-4ce4-87eb-70c49cc454be admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:22.281096 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256fa2f-9d49-4ce4-87eb-70c49cc454be admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:22.282087 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256fa2f-9d49-4ce4-87eb-70c49cc454be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:22.295790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256fa2f-9d49-4ce4-87eb-70c49cc454be admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:22.327984 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed52efc-c33b-480b-8875-9f473f6ed97e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:22.329686 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 909/1833] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:22.343360 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed52efc-c33b-480b-8875-9f473f6ed97e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:22.363942 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e427f-2ab3-4e7c-8c51-cfe62d09db2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:22.365056 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e427f-2ab3-4e7c-8c51-cfe62d09db2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:22.365548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e427f-2ab3-4e7c-8c51-cfe62d09db2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:22.366024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e427f-2ab3-4e7c-8c51-cfe62d09db2b admin admin] PATH_INFO: `/v3/projects/214d78790c914d52b15eef38aff7dedc/users/82bea38f6f97467c8d47aef2a706d43b/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:22.396846 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e427f-2ab3-4e7c-8c51-cfe62d09db2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=214d78790c914d52b15eef38aff7dedc, user_id=82bea38f6f97467c8d47aef2a706d43b, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:22.400904 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e427f-2ab3-4e7c-8c51-cfe62d09db2b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:22.511749 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 925/1834] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:22 2022] PUT /identity/v3/projects/214d78790c914d52b15eef38aff7dedc/users/82bea38f6f97467c8d47aef2a706d43b/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:22.522297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256fa2f-9d49-4ce4-87eb-70c49cc454be admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:22.534755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256fa2f-9d49-4ce4-87eb-70c49cc454be admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:22.539157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:22.610313 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc1d9f-bc31-4a4e-9b74-0218b4e027af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:22.611465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc1d9f-bc31-4a4e-9b74-0218b4e027af admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:22.611923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc1d9f-bc31-4a4e-9b74-0218b4e027af admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:22.612572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc1d9f-bc31-4a4e-9b74-0218b4e027af admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:22.613307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc1d9f-bc31-4a4e-9b74-0218b4e027af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:22.615833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc1d9f-bc31-4a4e-9b74-0218b4e027af admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:22.634819 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 910/1835] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:14:22.639338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e427f-2ab3-4e7c-8c51-cfe62d09db2b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:22.650591 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e427f-2ab3-4e7c-8c51-cfe62d09db2b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:22.663896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c229b-1d7c-448f-8772-bf0dfe12ffeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:22.663896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c229b-1d7c-448f-8772-bf0dfe12ffeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:22.663896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c229b-1d7c-448f-8772-bf0dfe12ffeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:22.663896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c229b-1d7c-448f-8772-bf0dfe12ffeb admin admin] PATH_INFO: `/v3/projects/214d78790c914d52b15eef38aff7dedc/users/82bea38f6f97467c8d47aef2a706d43b/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:22.665899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c229b-1d7c-448f-8772-bf0dfe12ffeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=214d78790c914d52b15eef38aff7dedc, user_id=82bea38f6f97467c8d47aef2a706d43b, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:22.669610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c229b-1d7c-448f-8772-bf0dfe12ffeb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:22.684772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 926/1836] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:22 2022] PUT /identity/v3/projects/214d78790c914d52b15eef38aff7dedc/users/82bea38f6f97467c8d47aef2a706d43b/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:14:22.697972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cf8ed-8743-4378-8f4c-8247af8842e6 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:22.698406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cf8ed-8743-4378-8f4c-8247af8842e6 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:22.698684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cf8ed-8743-4378-8f4c-8247af8842e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:22.762425 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-133cf8ed-8743-4378-8f4c-8247af8842e6 None None] MFA Rules not processed for user `82bea38f6f97467c8d47aef2a706d43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:22.765364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-133cf8ed-8743-4378-8f4c-8247af8842e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:22.923928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 911/1837] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:22.936424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c229b-1d7c-448f-8772-bf0dfe12ffeb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:22.938895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2f6c229b-1d7c-448f-8772-bf0dfe12ffeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:23.045256 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c229b-1d7c-448f-8772-bf0dfe12ffeb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:23.053377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484966e-f04a-436b-9010-e965eb6aafa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:23.053377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484966e-f04a-436b-9010-e965eb6aafa3 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:23.053377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484966e-f04a-436b-9010-e965eb6aafa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:23.053377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484966e-f04a-436b-9010-e965eb6aafa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:23.065836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484966e-f04a-436b-9010-e965eb6aafa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:23.067434 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484966e-f04a-436b-9010-e965eb6aafa3 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:23.080406 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 927/1838] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:14:24.568280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cf8ed-8743-4378-8f4c-8247af8842e6 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:24.569723 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-133cf8ed-8743-4378-8f4c-8247af8842e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:24.581505 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cf8ed-8743-4378-8f4c-8247af8842e6 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:24.585077 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e26c93-9f60-4d93-bf8e-0e27a63c10eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:24.585685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e26c93-9f60-4d93-bf8e-0e27a63c10eb service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:24.585960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e26c93-9f60-4d93-bf8e-0e27a63c10eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:24.586281 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e26c93-9f60-4d93-bf8e-0e27a63c10eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:24.653644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e26c93-9f60-4d93-bf8e-0e27a63c10eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:24.655869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e26c93-9f60-4d93-bf8e-0e27a63c10eb service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:24.668140 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 912/1839] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:24 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:14:33.074980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=74,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:34.654761 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484966e-f04a-436b-9010-e965eb6aafa3 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:34.662357 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=88 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:34.663620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484966e-f04a-436b-9010-e965eb6aafa3 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:34.668583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a32207-f9f6-4b22-9e80-cc045e533150 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:34.669362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a32207-f9f6-4b22-9e80-cc045e533150 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:34.669639 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a32207-f9f6-4b22-9e80-cc045e533150 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:34.669906 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a32207-f9f6-4b22-9e80-cc045e533150 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:34.731579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a32207-f9f6-4b22-9e80-cc045e533150 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:34.733248 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a32207-f9f6-4b22-9e80-cc045e533150 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:34.742957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 928/1840] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:14:36.292396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e26c93-9f60-4d93-bf8e-0e27a63c10eb service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:36.294852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-06e26c93-9f60-4d93-bf8e-0e27a63c10eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:36.394896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e26c93-9f60-4d93-bf8e-0e27a63c10eb service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:36.399915 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d86b3-cb22-4bbd-8db8-19f48acfb65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:36.401313 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d86b3-cb22-4bbd-8db8-19f48acfb65c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:36.401594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d86b3-cb22-4bbd-8db8-19f48acfb65c admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:36.401859 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d86b3-cb22-4bbd-8db8-19f48acfb65c admin admin] PATH_INFO: `/v3/users/5fb488cb6e2d443db89acfb629501d82` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:36.403067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d86b3-cb22-4bbd-8db8-19f48acfb65c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb488cb6e2d443db89acfb629501d82)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:36.405119 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d86b3-cb22-4bbd-8db8-19f48acfb65c admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:36.636573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b35d86b3-cb22-4bbd-8db8-19f48acfb65c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb488cb6e2d443db89acfb629501d82'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:36.637822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b35d86b3-cb22-4bbd-8db8-19f48acfb65c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb488cb6e2d443db89acfb629501d82'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:36.650924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b35d86b3-cb22-4bbd-8db8-19f48acfb65c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb488cb6e2d443db89acfb629501d82'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:36.728907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b35d86b3-cb22-4bbd-8db8-19f48acfb65c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb488cb6e2d443db89acfb629501d82'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:36.729562 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b35d86b3-cb22-4bbd-8db8-19f48acfb65c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb488cb6e2d443db89acfb629501d82'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:36.747018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 913/1841] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:36 2022] DELETE /identity/v3/users/5fb488cb6e2d443db89acfb629501d82 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:36.751659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b680d-34db-4186-aea9-97d4c525474d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:36.752172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b680d-34db-4186-aea9-97d4c525474d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:36.752278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b680d-34db-4186-aea9-97d4c525474d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:36.776383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7c7b680d-34db-4186-aea9-97d4c525474d None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:36.777858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7c7b680d-34db-4186-aea9-97d4c525474d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:36.852184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 929/1842] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:36.862152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d86b3-cb22-4bbd-8db8-19f48acfb65c admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:36.863969 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b35d86b3-cb22-4bbd-8db8-19f48acfb65c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:36.925344 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d86b3-cb22-4bbd-8db8-19f48acfb65c admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:36.929021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6b084-35c4-4275-a9d6-44e73baf6e2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:36.929627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6b084-35c4-4275-a9d6-44e73baf6e2d service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:36.929928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6b084-35c4-4275-a9d6-44e73baf6e2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:36.930180 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6b084-35c4-4275-a9d6-44e73baf6e2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:36.945790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6b084-35c4-4275-a9d6-44e73baf6e2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:36.948024 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb6b084-35c4-4275-a9d6-44e73baf6e2d service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:36.985672 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 914/1843] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:14:37.877753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b680d-34db-4186-aea9-97d4c525474d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:37.881626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7c7b680d-34db-4186-aea9-97d4c525474d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:37.909583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b680d-34db-4186-aea9-97d4c525474d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:37.930054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974e4f7-e007-4c3c-a248-cc387063b299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:37.930054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974e4f7-e007-4c3c-a248-cc387063b299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:37.930054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974e4f7-e007-4c3c-a248-cc387063b299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:37.930054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974e4f7-e007-4c3c-a248-cc387063b299 admin admin] PATH_INFO: `/v3/projects/4a1c57aa8b7f4dd0bcf397ff84097284` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:37.930726 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974e4f7-e007-4c3c-a248-cc387063b299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a1c57aa8b7f4dd0bcf397ff84097284)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:37.930726 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974e4f7-e007-4c3c-a248-cc387063b299 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:38.275788 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d974e4f7-e007-4c3c-a248-cc387063b299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a1c57aa8b7f4dd0bcf397ff84097284', 'request_id': 'req-d974e4f7-e007-4c3c-a248-cc387063b299'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:38.282520 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d974e4f7-e007-4c3c-a248-cc387063b299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a1c57aa8b7f4dd0bcf397ff84097284', 'request_id': 'req-d974e4f7-e007-4c3c-a248-cc387063b299'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:38.410336 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d974e4f7-e007-4c3c-a248-cc387063b299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a1c57aa8b7f4dd0bcf397ff84097284', 'request_id': 'req-d974e4f7-e007-4c3c-a248-cc387063b299'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:38.421515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d974e4f7-e007-4c3c-a248-cc387063b299 admin admin] The token cache is being invalidate because project 4a1c57aa8b7f4dd0bcf397ff84097284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:38.421802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d974e4f7-e007-4c3c-a248-cc387063b299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:38.422445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d974e4f7-e007-4c3c-a248-cc387063b299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:38.423985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 930/1844] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:37 2022] DELETE /identity/v3/projects/4a1c57aa8b7f4dd0bcf397ff84097284 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:38.428025 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6b084-35c4-4275-a9d6-44e73baf6e2d service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:38.437938 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6b084-35c4-4275-a9d6-44e73baf6e2d service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:38.456498 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:38.494695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83c020-05f6-40c3-adf6-4ca89e273532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:38.495346 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83c020-05f6-40c3-adf6-4ca89e273532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:38.495620 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83c020-05f6-40c3-adf6-4ca89e273532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:38.495887 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83c020-05f6-40c3-adf6-4ca89e273532 admin admin] PATH_INFO: `/v3/users/f11014add5d64419b506ad4a1b3647c6` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:38.497003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83c020-05f6-40c3-adf6-4ca89e273532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f11014add5d64419b506ad4a1b3647c6)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:38.498920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83c020-05f6-40c3-adf6-4ca89e273532 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:38.767284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-cb83c020-05f6-40c3-adf6-4ca89e273532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f11014add5d64419b506ad4a1b3647c6'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:38.767970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-cb83c020-05f6-40c3-adf6-4ca89e273532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f11014add5d64419b506ad4a1b3647c6'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:38.780477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-cb83c020-05f6-40c3-adf6-4ca89e273532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f11014add5d64419b506ad4a1b3647c6'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:38.856368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-cb83c020-05f6-40c3-adf6-4ca89e273532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f11014add5d64419b506ad4a1b3647c6'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:38.857099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-cb83c020-05f6-40c3-adf6-4ca89e273532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f11014add5d64419b506ad4a1b3647c6'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:38.876013 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 915/1845] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:38 2022] DELETE /identity/v3/users/f11014add5d64419b506ad4a1b3647c6 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:39.444569 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974e4f7-e007-4c3c-a248-cc387063b299 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:39.445997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d974e4f7-e007-4c3c-a248-cc387063b299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:39.471859 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974e4f7-e007-4c3c-a248-cc387063b299 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:39.507578 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:39.567069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e816cb-1d14-450f-91b9-14f3c76db3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:39.568087 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e816cb-1d14-450f-91b9-14f3c76db3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:39.568363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e816cb-1d14-450f-91b9-14f3c76db3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:39.568893 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e816cb-1d14-450f-91b9-14f3c76db3c2 admin admin] PATH_INFO: `/v3/projects/d56a5f5ad43842ff8e560562be508578` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:39.570054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e816cb-1d14-450f-91b9-14f3c76db3c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56a5f5ad43842ff8e560562be508578)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:39.572770 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e816cb-1d14-450f-91b9-14f3c76db3c2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:39.621819 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83c020-05f6-40c3-adf6-4ca89e273532 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:39.623620 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-cb83c020-05f6-40c3-adf6-4ca89e273532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:39.637899 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83c020-05f6-40c3-adf6-4ca89e273532 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:39.642481 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146faca3-99da-4539-b71b-95da3994f682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:39.643211 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146faca3-99da-4539-b71b-95da3994f682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:39.643489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146faca3-99da-4539-b71b-95da3994f682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:39.643761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146faca3-99da-4539-b71b-95da3994f682 admin admin] PATH_INFO: `/v3/users/55717b0e43fb4618b92a668a36c3bb63` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:39.644867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146faca3-99da-4539-b71b-95da3994f682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55717b0e43fb4618b92a668a36c3bb63)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:39.646792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146faca3-99da-4539-b71b-95da3994f682 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:39.708209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-70e816cb-1d14-450f-91b9-14f3c76db3c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56a5f5ad43842ff8e560562be508578', 'request_id': 'req-70e816cb-1d14-450f-91b9-14f3c76db3c2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:39.714762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-70e816cb-1d14-450f-91b9-14f3c76db3c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56a5f5ad43842ff8e560562be508578', 'request_id': 'req-70e816cb-1d14-450f-91b9-14f3c76db3c2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:39.798959 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-70e816cb-1d14-450f-91b9-14f3c76db3c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56a5f5ad43842ff8e560562be508578', 'request_id': 'req-70e816cb-1d14-450f-91b9-14f3c76db3c2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:39.810622 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-70e816cb-1d14-450f-91b9-14f3c76db3c2 admin admin] The token cache is being invalidate because project d56a5f5ad43842ff8e560562be508578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:39.810909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-70e816cb-1d14-450f-91b9-14f3c76db3c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:39.811621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-70e816cb-1d14-450f-91b9-14f3c76db3c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:39.813206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 931/1846] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:39 2022] DELETE /identity/v3/projects/d56a5f5ad43842ff8e560562be508578 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:39.818004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f4443-970a-4f5c-8b6e-f9e3300a2fa6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:39.818284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f4443-970a-4f5c-8b6e-f9e3300a2fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:39.818554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f4443-970a-4f5c-8b6e-f9e3300a2fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:39.839165 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e66f4443-970a-4f5c-8b6e-f9e3300a2fa6 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:39.840413 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e66f4443-970a-4f5c-8b6e-f9e3300a2fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:39.870432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-146faca3-99da-4539-b71b-95da3994f682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55717b0e43fb4618b92a668a36c3bb63'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:39.871447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-146faca3-99da-4539-b71b-95da3994f682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55717b0e43fb4618b92a668a36c3bb63'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:39.887374 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-146faca3-99da-4539-b71b-95da3994f682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55717b0e43fb4618b92a668a36c3bb63'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:39.905203 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 932/1847] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:39.913514 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196db98a-28bd-4756-ba96-11ef5ea51573 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:39.913799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196db98a-28bd-4756-ba96-11ef5ea51573 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:39.914106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196db98a-28bd-4756-ba96-11ef5ea51573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:39.935404 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-196db98a-28bd-4756-ba96-11ef5ea51573 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:39.937727 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-196db98a-28bd-4756-ba96-11ef5ea51573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:39.945635 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 933/1848] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:39.950683 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196db98a-28bd-4756-ba96-11ef5ea51573 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:39.959890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196db98a-28bd-4756-ba96-11ef5ea51573 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:39.965501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e041d76-4a6c-458c-a937-f03332f215c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:39.966095 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e041d76-4a6c-458c-a937-f03332f215c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:39.966370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e041d76-4a6c-458c-a937-f03332f215c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:39.966638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e041d76-4a6c-458c-a937-f03332f215c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:39.967309 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e041d76-4a6c-458c-a937-f03332f215c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:14:39.967606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e041d76-4a6c-458c-a937-f03332f215c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:39.969198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e041d76-4a6c-458c-a937-f03332f215c9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:39.981537 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 934/1849] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 08:14:39.981928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-146faca3-99da-4539-b71b-95da3994f682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55717b0e43fb4618b92a668a36c3bb63'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:39.982686 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-146faca3-99da-4539-b71b-95da3994f682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55717b0e43fb4618b92a668a36c3bb63'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:39.986401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e041d76-4a6c-458c-a937-f03332f215c9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:39.997585 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 916/1850] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:39 2022] DELETE /identity/v3/users/55717b0e43fb4618b92a668a36c3bb63 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:14:40.002113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d0ac1-b644-4630-8070-39e795f4b71c None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:40.002468 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d0ac1-b644-4630-8070-39e795f4b71c None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:40.009834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d0ac1-b644-4630-8070-39e795f4b71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:40.010007 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e041d76-4a6c-458c-a937-f03332f215c9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:40.025901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-4b0d0ac1-b644-4630-8070-39e795f4b71c None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:40.027591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4b0d0ac1-b644-4630-8070-39e795f4b71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:40.046924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:40.054439 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:40.092307 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a90a5f-8bf8-450a-a672-ce7e8ea7dfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:40.092935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a90a5f-8bf8-450a-a672-ce7e8ea7dfcc admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:40.093213 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a90a5f-8bf8-450a-a672-ce7e8ea7dfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:40.093463 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a90a5f-8bf8-450a-a672-ce7e8ea7dfcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:40.094102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a90a5f-8bf8-450a-a672-ce7e8ea7dfcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:40.096661 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a90a5f-8bf8-450a-a672-ce7e8ea7dfcc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:40.128540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 917/1851] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:40.139867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d0ac1-b644-4630-8070-39e795f4b71c None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:40.141471 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4b0d0ac1-b644-4630-8070-39e795f4b71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:40.231911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d0ac1-b644-4630-8070-39e795f4b71c None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:40.236214 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f57d8-9329-4746-a5f7-d21c00a14465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:40.237383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f57d8-9329-4746-a5f7-d21c00a14465 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:40.237721 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f57d8-9329-4746-a5f7-d21c00a14465 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:40.238149 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f57d8-9329-4746-a5f7-d21c00a14465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:40.253618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f57d8-9329-4746-a5f7-d21c00a14465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:40.256373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f57d8-9329-4746-a5f7-d21c00a14465 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:40.275860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 935/1852] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:39 2022] POST /identity/v3/projects => generated 401 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:14:40.286623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a90a5f-8bf8-450a-a672-ce7e8ea7dfcc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:40.287982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 918/1853] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:14:40.368647 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a90a5f-8bf8-450a-a672-ce7e8ea7dfcc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:40.375760 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bfab9-8998-4076-8472-6d936ab6bec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:40.376858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bfab9-8998-4076-8472-6d936ab6bec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:40.377328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bfab9-8998-4076-8472-6d936ab6bec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:40.377757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bfab9-8998-4076-8472-6d936ab6bec8 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:40.378922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1bfab9-8998-4076-8472-6d936ab6bec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:40.388270 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1bfab9-8998-4076-8472-6d936ab6bec8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:40.485929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 936/1854] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:40 2022] POST /identity/v3/users => generated 301 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:14:40.490813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f57d8-9329-4746-a5f7-d21c00a14465 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:40.524621 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f57d8-9329-4746-a5f7-d21c00a14465 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:40.540728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b74d8e-7eb3-499b-803f-734ba70e416d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:40.540728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b74d8e-7eb3-499b-803f-734ba70e416d admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:40.540728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b74d8e-7eb3-499b-803f-734ba70e416d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:40.540728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b74d8e-7eb3-499b-803f-734ba70e416d admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:40.541075 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b74d8e-7eb3-499b-803f-734ba70e416d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:40.541836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b74d8e-7eb3-499b-803f-734ba70e416d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:40.560394 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 919/1855] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:40.563885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bfab9-8998-4076-8472-6d936ab6bec8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:40.575716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bfab9-8998-4076-8472-6d936ab6bec8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:40.597418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952eacc4-ba6d-4301-887d-ac47146914c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:40.598053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eacc4-ba6d-4301-887d-ac47146914c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:40.598340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eacc4-ba6d-4301-887d-ac47146914c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:40.600447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eacc4-ba6d-4301-887d-ac47146914c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:40.602146 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eacc4-ba6d-4301-887d-ac47146914c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:40.613672 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eacc4-ba6d-4301-887d-ac47146914c5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:40.653133 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 937/1856] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:40.654313 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b74d8e-7eb3-499b-803f-734ba70e416d admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:40.676729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b74d8e-7eb3-499b-803f-734ba70e416d admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:40.717639 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692de678-95bd-4c08-9bba-ed8e6bc8567a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:40.720614 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692de678-95bd-4c08-9bba-ed8e6bc8567a admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:40.722292 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692de678-95bd-4c08-9bba-ed8e6bc8567a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:40.723857 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692de678-95bd-4c08-9bba-ed8e6bc8567a admin admin] PATH_INFO: `/v3/projects/aa257667f4b54140ade2962bf1b8a5c4/users/87dfa73cede746dea9a677b125d498f8/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:40.750328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692de678-95bd-4c08-9bba-ed8e6bc8567a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa257667f4b54140ade2962bf1b8a5c4, user_id=87dfa73cede746dea9a677b125d498f8, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:40.755724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692de678-95bd-4c08-9bba-ed8e6bc8567a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:40.839733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 920/1857] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:40 2022] PUT /identity/v3/projects/aa257667f4b54140ade2962bf1b8a5c4/users/87dfa73cede746dea9a677b125d498f8/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:14:40.852436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260e4a6-da13-43c2-b929-9350ade1a343 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:40.852568 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260e4a6-da13-43c2-b929-9350ade1a343 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:40.854652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260e4a6-da13-43c2-b929-9350ade1a343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:40.886537 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f260e4a6-da13-43c2-b929-9350ade1a343 None None] MFA Rules not processed for user `87dfa73cede746dea9a677b125d498f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:40.887978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f260e4a6-da13-43c2-b929-9350ade1a343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:40.891468 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692de678-95bd-4c08-9bba-ed8e6bc8567a admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:40.893799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-692de678-95bd-4c08-9bba-ed8e6bc8567a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:40.972151 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 938/1858] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:40 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:40.976504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692de678-95bd-4c08-9bba-ed8e6bc8567a admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:40.979509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260e4a6-da13-43c2-b929-9350ade1a343 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:40.989197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49820ba9-b71c-4898-b143-4b1811663f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:40.990199 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49820ba9-b71c-4898-b143-4b1811663f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:40.990770 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49820ba9-b71c-4898-b143-4b1811663f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:40.991266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49820ba9-b71c-4898-b143-4b1811663f32 admin admin] PATH_INFO: `/v3/users/2cd9d816179d407e819a69809837a595` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:40.992909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49820ba9-b71c-4898-b143-4b1811663f32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cd9d816179d407e819a69809837a595)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:40.994208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260e4a6-da13-43c2-b929-9350ade1a343 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:40.996092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49820ba9-b71c-4898-b143-4b1811663f32 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:40.999285 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3e4ef-53d6-48f0-97d4-12e456c28499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:40.999905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3e4ef-53d6-48f0-97d4-12e456c28499 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:41.000185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3e4ef-53d6-48f0-97d4-12e456c28499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:41.000458 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3e4ef-53d6-48f0-97d4-12e456c28499 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:41.001106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3e4ef-53d6-48f0-97d4-12e456c28499 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:41.003628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3e4ef-53d6-48f0-97d4-12e456c28499 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:41.023706 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:41.160659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 939/1859] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:40 2022] POST /identity/v3/projects => generated 399 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:41.164738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3e4ef-53d6-48f0-97d4-12e456c28499 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:41.176084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:41.226720 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-49820ba9-b71c-4898-b143-4b1811663f32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cd9d816179d407e819a69809837a595'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:41.227547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-49820ba9-b71c-4898-b143-4b1811663f32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cd9d816179d407e819a69809837a595'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:41.243811 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-49820ba9-b71c-4898-b143-4b1811663f32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cd9d816179d407e819a69809837a595'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:41.246819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3e4ef-53d6-48f0-97d4-12e456c28499 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:41.254595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27a6e5-9918-4c23-a967-f00ecf236edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:41.254709 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27a6e5-9918-4c23-a967-f00ecf236edb admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:41.255492 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27a6e5-9918-4c23-a967-f00ecf236edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:41.255492 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27a6e5-9918-4c23-a967-f00ecf236edb admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:41.256108 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27a6e5-9918-4c23-a967-f00ecf236edb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:41.258067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27a6e5-9918-4c23-a967-f00ecf236edb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:41.342455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-49820ba9-b71c-4898-b143-4b1811663f32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cd9d816179d407e819a69809837a595'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:41.343530 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-49820ba9-b71c-4898-b143-4b1811663f32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cd9d816179d407e819a69809837a595'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:41.358738 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 921/1860] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:40 2022] DELETE /identity/v3/users/2cd9d816179d407e819a69809837a595 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:41.362967 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4da00-d929-4aee-97bd-ea9e2eb0bf63 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:41.363281 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4da00-d929-4aee-97bd-ea9e2eb0bf63 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:41.363928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4da00-d929-4aee-97bd-ea9e2eb0bf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:41.403274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-a6c4da00-d929-4aee-97bd-ea9e2eb0bf63 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:41.404616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a6c4da00-d929-4aee-97bd-ea9e2eb0bf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:41.426182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 940/1861] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:41 2022] POST /identity/v3/users => generated 306 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:41.427817 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27a6e5-9918-4c23-a967-f00ecf236edb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:41.429477 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6c27a6e5-9918-4c23-a967-f00ecf236edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:41.449809 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27a6e5-9918-4c23-a967-f00ecf236edb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:41.526594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 922/1862] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:41.536504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4da00-d929-4aee-97bd-ea9e2eb0bf63 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:41.538049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a6c4da00-d929-4aee-97bd-ea9e2eb0bf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:41.558090 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3619f6-060e-4d3a-a315-654ad44c46c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:41.558855 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3619f6-060e-4d3a-a315-654ad44c46c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:41.559267 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3619f6-060e-4d3a-a315-654ad44c46c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:41.559641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3619f6-060e-4d3a-a315-654ad44c46c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:41.560396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3619f6-060e-4d3a-a315-654ad44c46c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:41.562334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3619f6-060e-4d3a-a315-654ad44c46c2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:41.574987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 941/1863] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:14:41.579074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3619f6-060e-4d3a-a315-654ad44c46c2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:41.590486 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3619f6-060e-4d3a-a315-654ad44c46c2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:41.595348 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec5749-19c1-41c0-bb41-e9148ab8cb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:41.596147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec5749-19c1-41c0-bb41-e9148ab8cb1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:41.596678 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec5749-19c1-41c0-bb41-e9148ab8cb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:41.597020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec5749-19c1-41c0-bb41-e9148ab8cb1e admin admin] PATH_INFO: `/v3/projects/a7df74a97ef340fa99fe62b57628d529/users/eda467a493484b32b9e42d0597aaa2a3/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:41.608824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4da00-d929-4aee-97bd-ea9e2eb0bf63 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:41.613377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf2bdf-0b2c-42b2-9fd9-eb85755b7547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:41.614039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf2bdf-0b2c-42b2-9fd9-eb85755b7547 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:41.614274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf2bdf-0b2c-42b2-9fd9-eb85755b7547 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:41.614547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf2bdf-0b2c-42b2-9fd9-eb85755b7547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:41.615472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec5749-19c1-41c0-bb41-e9148ab8cb1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7df74a97ef340fa99fe62b57628d529, user_id=eda467a493484b32b9e42d0597aaa2a3, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:41.617583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec5749-19c1-41c0-bb41-e9148ab8cb1e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:41.626100 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf2bdf-0b2c-42b2-9fd9-eb85755b7547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:41.630802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf2bdf-0b2c-42b2-9fd9-eb85755b7547 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:41.639048 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 923/1864] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:14:41.639895 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf2bdf-0b2c-42b2-9fd9-eb85755b7547 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:41.641242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0dcf2bdf-0b2c-42b2-9fd9-eb85755b7547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:41.657793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf2bdf-0b2c-42b2-9fd9-eb85755b7547 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:41.663010 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89508c7-1cf9-4f57-a540-a734e41577e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:41.663878 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89508c7-1cf9-4f57-a540-a734e41577e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:41.664320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89508c7-1cf9-4f57-a540-a734e41577e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:41.664756 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89508c7-1cf9-4f57-a540-a734e41577e9 admin admin] PATH_INFO: `/v3/projects/57781ed5ea154c51b4641282fbf0f444` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:41.666282 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89508c7-1cf9-4f57-a540-a734e41577e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57781ed5ea154c51b4641282fbf0f444)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:41.668419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89508c7-1cf9-4f57-a540-a734e41577e9 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:41.719801 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 942/1865] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:41 2022] PUT /identity/v3/projects/a7df74a97ef340fa99fe62b57628d529/users/eda467a493484b32b9e42d0597aaa2a3/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:41.723698 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec5749-19c1-41c0-bb41-e9148ab8cb1e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:41.808032 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec5749-19c1-41c0-bb41-e9148ab8cb1e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:41.814069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1e789-95fa-4539-89ff-1537db71126c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:41.814680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1e789-95fa-4539-89ff-1537db71126c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:41.814970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1e789-95fa-4539-89ff-1537db71126c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:41.815245 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1e789-95fa-4539-89ff-1537db71126c admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:41.815909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1e789-95fa-4539-89ff-1537db71126c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:41.818329 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1e789-95fa-4539-89ff-1537db71126c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:41.835716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 943/1866] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:14:41.836615 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1e789-95fa-4539-89ff-1537db71126c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:41.845407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:41.850139 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1e789-95fa-4539-89ff-1537db71126c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:41.853173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:41.880616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c89508c7-1cf9-4f57-a540-a734e41577e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57781ed5ea154c51b4641282fbf0f444', 'request_id': 'req-c89508c7-1cf9-4f57-a540-a734e41577e9'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:41.904695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c89508c7-1cf9-4f57-a540-a734e41577e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57781ed5ea154c51b4641282fbf0f444', 'request_id': 'req-c89508c7-1cf9-4f57-a540-a734e41577e9'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:41.935470 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e19e96-9082-4fcf-be72-6dc2626472a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:41.937049 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e19e96-9082-4fcf-be72-6dc2626472a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:41.937049 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e19e96-9082-4fcf-be72-6dc2626472a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:41.937049 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e19e96-9082-4fcf-be72-6dc2626472a9 admin admin] PATH_INFO: `/v3/projects/a7df74a97ef340fa99fe62b57628d529/users/eda467a493484b32b9e42d0597aaa2a3/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:41.939670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e19e96-9082-4fcf-be72-6dc2626472a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7df74a97ef340fa99fe62b57628d529, user_id=eda467a493484b32b9e42d0597aaa2a3, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:41.943022 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e19e96-9082-4fcf-be72-6dc2626472a9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:41.968207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 944/1867] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:41 2022] PUT /identity/v3/projects/a7df74a97ef340fa99fe62b57628d529/users/eda467a493484b32b9e42d0597aaa2a3/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:14:41.970073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b240cef-a929-4eff-94eb-7ac06b8cc55a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:41.970427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b240cef-a929-4eff-94eb-7ac06b8cc55a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:41.970772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b240cef-a929-4eff-94eb-7ac06b8cc55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:42.003004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6b240cef-a929-4eff-94eb-7ac06b8cc55a None None] MFA Rules not processed for user `eda467a493484b32b9e42d0597aaa2a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:42.006061 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6b240cef-a929-4eff-94eb-7ac06b8cc55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:42.030029 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c89508c7-1cf9-4f57-a540-a734e41577e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57781ed5ea154c51b4641282fbf0f444', 'request_id': 'req-c89508c7-1cf9-4f57-a540-a734e41577e9'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:42.053691 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c89508c7-1cf9-4f57-a540-a734e41577e9 admin admin] The token cache is being invalidate because project 57781ed5ea154c51b4641282fbf0f444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:42.054583 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c89508c7-1cf9-4f57-a540-a734e41577e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:42.055767 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c89508c7-1cf9-4f57-a540-a734e41577e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:42.059511 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 924/1868] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:41 2022] DELETE /identity/v3/projects/57781ed5ea154c51b4641282fbf0f444 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:42.063909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89508c7-1cf9-4f57-a540-a734e41577e9 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:42.066803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-c89508c7-1cf9-4f57-a540-a734e41577e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:42.087118 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 945/1869] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:41 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:42.094228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89508c7-1cf9-4f57-a540-a734e41577e9 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:42.097646 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b240cef-a929-4eff-94eb-7ac06b8cc55a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:42.099036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6b240cef-a929-4eff-94eb-7ac06b8cc55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:42.177787 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182aba6-a5e2-4452-b831-a8eb616cedfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:42.178435 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182aba6-a5e2-4452-b831-a8eb616cedfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:42.178718 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182aba6-a5e2-4452-b831-a8eb616cedfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:42.179246 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182aba6-a5e2-4452-b831-a8eb616cedfe admin admin] PATH_INFO: `/v3/users/8b4871d1281c46a89faaece68e21551c` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:42.181831 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182aba6-a5e2-4452-b831-a8eb616cedfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b4871d1281c46a89faaece68e21551c)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:42.184258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182aba6-a5e2-4452-b831-a8eb616cedfe admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:42.210916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b240cef-a929-4eff-94eb-7ac06b8cc55a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:42.229678 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a09cc6-3190-492f-b28c-583eeabfcede service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:42.230584 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a09cc6-3190-492f-b28c-583eeabfcede service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:42.231081 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a09cc6-3190-492f-b28c-583eeabfcede service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:42.231694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a09cc6-3190-492f-b28c-583eeabfcede service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:42.250586 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a09cc6-3190-492f-b28c-583eeabfcede service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:42.252773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a09cc6-3190-492f-b28c-583eeabfcede service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:42.265974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 946/1870] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:42 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:14:42.285500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a09cc6-3190-492f-b28c-583eeabfcede service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:42.285500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-24a09cc6-3190-492f-b28c-583eeabfcede service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:42.305661 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a09cc6-3190-492f-b28c-583eeabfcede service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:42.312909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6103854-9c74-4632-99da-e29931b92911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:42.313563 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6103854-9c74-4632-99da-e29931b92911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:42.313869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6103854-9c74-4632-99da-e29931b92911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:42.314139 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6103854-9c74-4632-99da-e29931b92911 admin admin] PATH_INFO: `/v3/projects/08dbba82e2b74248a34b366190b1199e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:42.315335 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6103854-9c74-4632-99da-e29931b92911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08dbba82e2b74248a34b366190b1199e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:42.317965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6103854-9c74-4632-99da-e29931b92911 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:42.344841 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:42.373776 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3182aba6-a5e2-4452-b831-a8eb616cedfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b4871d1281c46a89faaece68e21551c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:42.374827 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3182aba6-a5e2-4452-b831-a8eb616cedfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b4871d1281c46a89faaece68e21551c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:42.392015 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3182aba6-a5e2-4452-b831-a8eb616cedfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b4871d1281c46a89faaece68e21551c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:42.442873 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a6103854-9c74-4632-99da-e29931b92911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08dbba82e2b74248a34b366190b1199e', 'request_id': 'req-a6103854-9c74-4632-99da-e29931b92911'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:42.449755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a6103854-9c74-4632-99da-e29931b92911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08dbba82e2b74248a34b366190b1199e', 'request_id': 'req-a6103854-9c74-4632-99da-e29931b92911'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:42.467978 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3182aba6-a5e2-4452-b831-a8eb616cedfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b4871d1281c46a89faaece68e21551c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:42.468954 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3182aba6-a5e2-4452-b831-a8eb616cedfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b4871d1281c46a89faaece68e21551c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:42.491725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 925/1871] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:42 2022] DELETE /identity/v3/users/8b4871d1281c46a89faaece68e21551c => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:42.529516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182aba6-a5e2-4452-b831-a8eb616cedfe admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:42.532990 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3182aba6-a5e2-4452-b831-a8eb616cedfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:42.598795 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a6103854-9c74-4632-99da-e29931b92911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08dbba82e2b74248a34b366190b1199e', 'request_id': 'req-a6103854-9c74-4632-99da-e29931b92911'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:42.627826 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a6103854-9c74-4632-99da-e29931b92911 admin admin] The token cache is being invalidate because project 08dbba82e2b74248a34b366190b1199e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:42.628084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a6103854-9c74-4632-99da-e29931b92911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:42.630548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a6103854-9c74-4632-99da-e29931b92911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:42.632813 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 947/1872] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:42 2022] DELETE /identity/v3/projects/08dbba82e2b74248a34b366190b1199e => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:42.637668 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6103854-9c74-4632-99da-e29931b92911 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:42.645485 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a6103854-9c74-4632-99da-e29931b92911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:42.700522 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:42.713155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182aba6-a5e2-4452-b831-a8eb616cedfe admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:42.719158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346e240-4827-43ed-b58c-a9d6f9e54f52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:42.720895 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346e240-4827-43ed-b58c-a9d6f9e54f52 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:42.721341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346e240-4827-43ed-b58c-a9d6f9e54f52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:42.721821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346e240-4827-43ed-b58c-a9d6f9e54f52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:42.724203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e346e240-4827-43ed-b58c-a9d6f9e54f52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:42.810574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6103854-9c74-4632-99da-e29931b92911 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:42.893895 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346e240-4827-43ed-b58c-a9d6f9e54f52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:42.899686 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346e240-4827-43ed-b58c-a9d6f9e54f52 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:42.925644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 926/1873] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:42 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:14:42.944596 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8f567-ab3a-43f3-8c3e-23afe73689b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:42.944596 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8f567-ab3a-43f3-8c3e-23afe73689b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:42.944905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8f567-ab3a-43f3-8c3e-23afe73689b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:42.945000 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8f567-ab3a-43f3-8c3e-23afe73689b6 admin admin] PATH_INFO: `/v3/users/e20105e54e6f4d9b830929a95007a1ec` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:42.947686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8f567-ab3a-43f3-8c3e-23afe73689b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e20105e54e6f4d9b830929a95007a1ec)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:42.953896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8f567-ab3a-43f3-8c3e-23afe73689b6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:43.084244 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346e240-4827-43ed-b58c-a9d6f9e54f52 service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:43.086018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e346e240-4827-43ed-b58c-a9d6f9e54f52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:43.136780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346e240-4827-43ed-b58c-a9d6f9e54f52 service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:43.147293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b1b4c-5709-48de-a6f6-89c86733c503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:43.148181 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b1b4c-5709-48de-a6f6-89c86733c503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:43.148847 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b1b4c-5709-48de-a6f6-89c86733c503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:43.149309 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b1b4c-5709-48de-a6f6-89c86733c503 admin admin] PATH_INFO: `/v3/projects/41b63c8853a94fe78e935a3badcbf26e` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:43.154384 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b1b4c-5709-48de-a6f6-89c86733c503 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41b63c8853a94fe78e935a3badcbf26e)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:43.154384 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b1b4c-5709-48de-a6f6-89c86733c503 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:43.231605 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-62a8f567-ab3a-43f3-8c3e-23afe73689b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e20105e54e6f4d9b830929a95007a1ec'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:43.258110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:43.270626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-62a8f567-ab3a-43f3-8c3e-23afe73689b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e20105e54e6f4d9b830929a95007a1ec'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:43.271752 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-62a8f567-ab3a-43f3-8c3e-23afe73689b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e20105e54e6f4d9b830929a95007a1ec'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:43.436474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-62a8f567-ab3a-43f3-8c3e-23afe73689b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e20105e54e6f4d9b830929a95007a1ec'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:43.439315 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-62a8f567-ab3a-43f3-8c3e-23afe73689b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e20105e54e6f4d9b830929a95007a1ec'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:43.472399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 948/1874] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:42 2022] DELETE /identity/v3/users/e20105e54e6f4d9b830929a95007a1ec => generated 0 bytes in 835 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:43.489694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9d5b1b4c-5709-48de-a6f6-89c86733c503 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41b63c8853a94fe78e935a3badcbf26e', 'request_id': 'req-9d5b1b4c-5709-48de-a6f6-89c86733c503'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:43.503021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9d5b1b4c-5709-48de-a6f6-89c86733c503 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41b63c8853a94fe78e935a3badcbf26e', 'request_id': 'req-9d5b1b4c-5709-48de-a6f6-89c86733c503'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:43.576335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9d5b1b4c-5709-48de-a6f6-89c86733c503 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41b63c8853a94fe78e935a3badcbf26e', 'request_id': 'req-9d5b1b4c-5709-48de-a6f6-89c86733c503'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:43.590559 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9d5b1b4c-5709-48de-a6f6-89c86733c503 admin admin] The token cache is being invalidate because project 41b63c8853a94fe78e935a3badcbf26e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:43.591249 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9d5b1b4c-5709-48de-a6f6-89c86733c503 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:43.592070 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9d5b1b4c-5709-48de-a6f6-89c86733c503 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:43.595374 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 927/1875] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:43 2022] DELETE /identity/v3/projects/41b63c8853a94fe78e935a3badcbf26e => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:43.599781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8f567-ab3a-43f3-8c3e-23afe73689b6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:43.601847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-62a8f567-ab3a-43f3-8c3e-23afe73689b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:43.722934 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8f567-ab3a-43f3-8c3e-23afe73689b6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:43.730341 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528730df-4939-48d1-9ad3-39ddb8bfe876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:43.731682 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528730df-4939-48d1-9ad3-39ddb8bfe876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:43.731966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528730df-4939-48d1-9ad3-39ddb8bfe876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:43.732244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528730df-4939-48d1-9ad3-39ddb8bfe876 admin admin] PATH_INFO: `/v3/users/82bea38f6f97467c8d47aef2a706d43b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:43.734640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528730df-4939-48d1-9ad3-39ddb8bfe876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82bea38f6f97467c8d47aef2a706d43b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:43.736958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528730df-4939-48d1-9ad3-39ddb8bfe876 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:44.001732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-528730df-4939-48d1-9ad3-39ddb8bfe876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82bea38f6f97467c8d47aef2a706d43b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:44.026286 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-528730df-4939-48d1-9ad3-39ddb8bfe876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82bea38f6f97467c8d47aef2a706d43b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:44.028141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-528730df-4939-48d1-9ad3-39ddb8bfe876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82bea38f6f97467c8d47aef2a706d43b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:44.186324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-528730df-4939-48d1-9ad3-39ddb8bfe876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82bea38f6f97467c8d47aef2a706d43b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:44.186324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-528730df-4939-48d1-9ad3-39ddb8bfe876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82bea38f6f97467c8d47aef2a706d43b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:44.201254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 949/1876] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:43 2022] DELETE /identity/v3/users/82bea38f6f97467c8d47aef2a706d43b => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:44.869341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b1b4c-5709-48de-a6f6-89c86733c503 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:44.870805 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9d5b1b4c-5709-48de-a6f6-89c86733c503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:44.959838 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b1b4c-5709-48de-a6f6-89c86733c503 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:44.971271 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528730df-4939-48d1-9ad3-39ddb8bfe876 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:44.972779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-528730df-4939-48d1-9ad3-39ddb8bfe876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:45.011630 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:45.075619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328552f8-cfee-481a-9a2b-5384565c13cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:45.076751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328552f8-cfee-481a-9a2b-5384565c13cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:45.080822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328552f8-cfee-481a-9a2b-5384565c13cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:45.080822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328552f8-cfee-481a-9a2b-5384565c13cc admin admin] PATH_INFO: `/v3/projects/82be3d00897a480ca14060d4ced5f0be` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:45.080822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328552f8-cfee-481a-9a2b-5384565c13cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=82be3d00897a480ca14060d4ced5f0be)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:45.082093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328552f8-cfee-481a-9a2b-5384565c13cc admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:45.113671 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:45.132284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528730df-4939-48d1-9ad3-39ddb8bfe876 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:45.139724 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3feca-22b7-48f3-b387-69ce6ee51981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:45.140675 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3feca-22b7-48f3-b387-69ce6ee51981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:45.141237 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3feca-22b7-48f3-b387-69ce6ee51981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:45.141751 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3feca-22b7-48f3-b387-69ce6ee51981 admin admin] PATH_INFO: `/v3/projects/214d78790c914d52b15eef38aff7dedc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:45.143048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3feca-22b7-48f3-b387-69ce6ee51981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=214d78790c914d52b15eef38aff7dedc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:45.146154 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3feca-22b7-48f3-b387-69ce6ee51981 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:45.240730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-328552f8-cfee-481a-9a2b-5384565c13cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82be3d00897a480ca14060d4ced5f0be', 'request_id': 'req-328552f8-cfee-481a-9a2b-5384565c13cc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:45.272178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-328552f8-cfee-481a-9a2b-5384565c13cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82be3d00897a480ca14060d4ced5f0be', 'request_id': 'req-328552f8-cfee-481a-9a2b-5384565c13cc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:45.289587 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f4e3feca-22b7-48f3-b387-69ce6ee51981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '214d78790c914d52b15eef38aff7dedc', 'request_id': 'req-f4e3feca-22b7-48f3-b387-69ce6ee51981'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:45.299392 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f4e3feca-22b7-48f3-b387-69ce6ee51981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '214d78790c914d52b15eef38aff7dedc', 'request_id': 'req-f4e3feca-22b7-48f3-b387-69ce6ee51981'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:45.358120 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-328552f8-cfee-481a-9a2b-5384565c13cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82be3d00897a480ca14060d4ced5f0be', 'request_id': 'req-328552f8-cfee-481a-9a2b-5384565c13cc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:45.374115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-328552f8-cfee-481a-9a2b-5384565c13cc admin admin] The token cache is being invalidate because project 82be3d00897a480ca14060d4ced5f0be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:45.375398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-328552f8-cfee-481a-9a2b-5384565c13cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:45.377770 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-328552f8-cfee-481a-9a2b-5384565c13cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:45.381003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 928/1877] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:44 2022] DELETE /identity/v3/projects/82be3d00897a480ca14060d4ced5f0be => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:45.387320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450eb799-b4fa-4088-912c-6a0d1d4eca4a None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:45.388472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450eb799-b4fa-4088-912c-6a0d1d4eca4a None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:45.392971 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450eb799-b4fa-4088-912c-6a0d1d4eca4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:45.421661 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-450eb799-b4fa-4088-912c-6a0d1d4eca4a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:45.423151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-450eb799-b4fa-4088-912c-6a0d1d4eca4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:45.456526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f4e3feca-22b7-48f3-b387-69ce6ee51981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '214d78790c914d52b15eef38aff7dedc', 'request_id': 'req-f4e3feca-22b7-48f3-b387-69ce6ee51981'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:45.475449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f4e3feca-22b7-48f3-b387-69ce6ee51981 admin admin] The token cache is being invalidate because project 214d78790c914d52b15eef38aff7dedc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:45.475742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f4e3feca-22b7-48f3-b387-69ce6ee51981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:45.476602 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f4e3feca-22b7-48f3-b387-69ce6ee51981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:45.480382 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 950/1878] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:44 2022] DELETE /identity/v3/projects/214d78790c914d52b15eef38aff7dedc => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:45.490369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294dcd6-8239-4fa6-b872-cbe921974079 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:45.490719 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294dcd6-8239-4fa6-b872-cbe921974079 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:45.491016 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294dcd6-8239-4fa6-b872-cbe921974079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:45.534575 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e294dcd6-8239-4fa6-b872-cbe921974079 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:45.536118 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e294dcd6-8239-4fa6-b872-cbe921974079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:45.566293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 929/1879] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:45.574080 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54771974-3d5a-4bb8-8ec6-4b6b151a66d0 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:45.574445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54771974-3d5a-4bb8-8ec6-4b6b151a66d0 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:45.574733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54771974-3d5a-4bb8-8ec6-4b6b151a66d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:45.596858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-54771974-3d5a-4bb8-8ec6-4b6b151a66d0 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:45.598337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-54771974-3d5a-4bb8-8ec6-4b6b151a66d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:45.607745 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 930/1880] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:45.612527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54771974-3d5a-4bb8-8ec6-4b6b151a66d0 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:45.626106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 951/1881] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:45.626687 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54771974-3d5a-4bb8-8ec6-4b6b151a66d0 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:45.635236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f8f41-6091-44d9-a5c4-c73bff14beff None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:45.637451 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f8f41-6091-44d9-a5c4-c73bff14beff None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:45.637451 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f8f41-6091-44d9-a5c4-c73bff14beff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:45.638503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94169c-bbaf-429d-b0e3-37927bbe7b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:45.639058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94169c-bbaf-429d-b0e3-37927bbe7b57 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:45.639529 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94169c-bbaf-429d-b0e3-37927bbe7b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:45.639816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94169c-bbaf-429d-b0e3-37927bbe7b57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:45.640483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94169c-bbaf-429d-b0e3-37927bbe7b57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:14:45.640935 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94169c-bbaf-429d-b0e3-37927bbe7b57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:45.643058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94169c-bbaf-429d-b0e3-37927bbe7b57 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:45.657354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 931/1882] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 08:14:45.662063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94169c-bbaf-429d-b0e3-37927bbe7b57 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:45.676722 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2e7f8f41-6091-44d9-a5c4-c73bff14beff None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:45.677577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94169c-bbaf-429d-b0e3-37927bbe7b57 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:45.682406 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2e7f8f41-6091-44d9-a5c4-c73bff14beff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:45.684560 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5309a-3700-41a8-b84f-aeeb35c83f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:45.689070 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5309a-3700-41a8-b84f-aeeb35c83f9b admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:45.689070 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5309a-3700-41a8-b84f-aeeb35c83f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:45.689070 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5309a-3700-41a8-b84f-aeeb35c83f9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:45.689070 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5309a-3700-41a8-b84f-aeeb35c83f9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:45.689070 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5309a-3700-41a8-b84f-aeeb35c83f9b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:45.699646 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 952/1883] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:45.703711 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f8f41-6091-44d9-a5c4-c73bff14beff None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:45.716666 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f8f41-6091-44d9-a5c4-c73bff14beff None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:45.725823 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850c778-3908-47e9-b091-87acf5bd1637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:45.726663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850c778-3908-47e9-b091-87acf5bd1637 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:45.727676 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850c778-3908-47e9-b091-87acf5bd1637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:45.728105 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850c778-3908-47e9-b091-87acf5bd1637 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:45.729143 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850c778-3908-47e9-b091-87acf5bd1637 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:14:45.735573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850c778-3908-47e9-b091-87acf5bd1637 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:45.735573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850c778-3908-47e9-b091-87acf5bd1637 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:45.749784 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 953/1884] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:14:45.755287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850c778-3908-47e9-b091-87acf5bd1637 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:45.784045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850c778-3908-47e9-b091-87acf5bd1637 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:45.788834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 932/1885] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:45 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:45.792987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5309a-3700-41a8-b84f-aeeb35c83f9b admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:45.798175 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f14e05-9ad1-45fb-a92f-4f28b5c34e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:45.809242 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f14e05-9ad1-45fb-a92f-4f28b5c34e81 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:45.809242 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f14e05-9ad1-45fb-a92f-4f28b5c34e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:45.809242 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f14e05-9ad1-45fb-a92f-4f28b5c34e81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:45.809242 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f14e05-9ad1-45fb-a92f-4f28b5c34e81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:45.809242 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f14e05-9ad1-45fb-a92f-4f28b5c34e81 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:45.835313 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:45.903787 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5309a-3700-41a8-b84f-aeeb35c83f9b admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:45.907887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 954/1886] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:45 2022] POST /identity/v3/projects => generated 365 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:14:45.911638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f14e05-9ad1-45fb-a92f-4f28b5c34e81 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:46.000625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09037beb-6e14-4742-b458-5c8b3664695b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:46.001471 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09037beb-6e14-4742-b458-5c8b3664695b admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:46.001640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:46.002074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09037beb-6e14-4742-b458-5c8b3664695b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:46.002212 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09037beb-6e14-4742-b458-5c8b3664695b admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:46.002862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09037beb-6e14-4742-b458-5c8b3664695b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:46.005494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09037beb-6e14-4742-b458-5c8b3664695b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:46.023871 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f14e05-9ad1-45fb-a92f-4f28b5c34e81 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:46.031739 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f6e5e-333f-41b9-9add-e01f966eb323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:46.032514 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f6e5e-333f-41b9-9add-e01f966eb323 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:46.032859 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f6e5e-333f-41b9-9add-e01f966eb323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:46.033506 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f6e5e-333f-41b9-9add-e01f966eb323 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:46.034064 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f6e5e-333f-41b9-9add-e01f966eb323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:46.036385 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f6e5e-333f-41b9-9add-e01f966eb323 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:46.119132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 933/1887] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:45 2022] POST /identity/v3/users => generated 292 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:14:46.119132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09037beb-6e14-4742-b458-5c8b3664695b admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:46.137860 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09037beb-6e14-4742-b458-5c8b3664695b admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:46.160025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 955/1888] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:45 2022] POST /identity/v3/users => generated 283 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:14:46.164209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f6e5e-333f-41b9-9add-e01f966eb323 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:46.168149 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fba63-b77f-41f7-99b7-69bfaff0a2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:46.168783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fba63-b77f-41f7-99b7-69bfaff0a2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:46.169189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fba63-b77f-41f7-99b7-69bfaff0a2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:46.169378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fba63-b77f-41f7-99b7-69bfaff0a2c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:46.170093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fba63-b77f-41f7-99b7-69bfaff0a2c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:46.171782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fba63-b77f-41f7-99b7-69bfaff0a2c4 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:46.180352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f6e5e-333f-41b9-9add-e01f966eb323 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:46.193141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 934/1889] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:46.195734 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fba63-b77f-41f7-99b7-69bfaff0a2c4 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:46.196878 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ce465-9903-41ea-98d8-c544b8f0a922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:46.197623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ce465-9903-41ea-98d8-c544b8f0a922 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:46.197944 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ce465-9903-41ea-98d8-c544b8f0a922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:46.200156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ce465-9903-41ea-98d8-c544b8f0a922 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:46.200866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ce465-9903-41ea-98d8-c544b8f0a922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:46.202551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ce465-9903-41ea-98d8-c544b8f0a922 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:46.220464 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fba63-b77f-41f7-99b7-69bfaff0a2c4 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:46.232866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 956/1890] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:46.234844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ce465-9903-41ea-98d8-c544b8f0a922 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:46.238162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94b21b-a6b4-4fb0-ad88-9303c1538d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:46.238788 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94b21b-a6b4-4fb0-ad88-9303c1538d21 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:46.239136 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94b21b-a6b4-4fb0-ad88-9303c1538d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:46.239357 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94b21b-a6b4-4fb0-ad88-9303c1538d21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:46.240831 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94b21b-a6b4-4fb0-ad88-9303c1538d21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:46.246259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94b21b-a6b4-4fb0-ad88-9303c1538d21 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:46.261100 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ce465-9903-41ea-98d8-c544b8f0a922 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:46.281029 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 935/1891] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:46.287471 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94b21b-a6b4-4fb0-ad88-9303c1538d21 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:46.287643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c1140-bd7d-4d2f-8ec8-204ac6a078b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:46.287643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c1140-bd7d-4d2f-8ec8-204ac6a078b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:46.287643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c1140-bd7d-4d2f-8ec8-204ac6a078b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:46.287643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c1140-bd7d-4d2f-8ec8-204ac6a078b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:46.287935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c1140-bd7d-4d2f-8ec8-204ac6a078b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:46.289483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c1140-bd7d-4d2f-8ec8-204ac6a078b5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:46.309460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94b21b-a6b4-4fb0-ad88-9303c1538d21 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:46.322150 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 957/1892] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:46.323143 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c1140-bd7d-4d2f-8ec8-204ac6a078b5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:46.323957 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ea37f-c302-4cea-b38d-c87151f29a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:46.324743 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ea37f-c302-4cea-b38d-c87151f29a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:46.325127 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ea37f-c302-4cea-b38d-c87151f29a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:46.327846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ea37f-c302-4cea-b38d-c87151f29a6f admin admin] PATH_INFO: `/v3/projects/23ffa92fc476474bb1f95c75bf202cce/users/89b765738e884fd3a1cbb2e1409eca7a/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:46.344257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c1140-bd7d-4d2f-8ec8-204ac6a078b5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:46.359519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e260fbb-40e6-4719-88c7-1f131f4b8957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:46.360201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e260fbb-40e6-4719-88c7-1f131f4b8957 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:46.360474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e260fbb-40e6-4719-88c7-1f131f4b8957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:46.360661 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ea37f-c302-4cea-b38d-c87151f29a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ffa92fc476474bb1f95c75bf202cce, user_id=89b765738e884fd3a1cbb2e1409eca7a, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:46.360774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e260fbb-40e6-4719-88c7-1f131f4b8957 admin admin] PATH_INFO: `/v3/projects/6822b951e5b445a09a05693b1b1cf209/users/1a968944f4bc4c5b93f98d124b88fe01/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:46.363774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ea37f-c302-4cea-b38d-c87151f29a6f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:46.395738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e260fbb-40e6-4719-88c7-1f131f4b8957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6822b951e5b445a09a05693b1b1cf209, user_id=1a968944f4bc4c5b93f98d124b88fe01, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:46.398633 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e260fbb-40e6-4719-88c7-1f131f4b8957 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:46.766982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 936/1893] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:46 2022] PUT /identity/v3/projects/23ffa92fc476474bb1f95c75bf202cce/users/89b765738e884fd3a1cbb2e1409eca7a/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:14:46.777577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3c07b-9813-4759-a7db-fb11f6255187 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:46.777857 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3c07b-9813-4759-a7db-fb11f6255187 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:46.778133 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3c07b-9813-4759-a7db-fb11f6255187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:46.830975 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 958/1894] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:46 2022] PUT /identity/v3/projects/6822b951e5b445a09a05693b1b1cf209/users/1a968944f4bc4c5b93f98d124b88fe01/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:46.831205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-9de3c07b-9813-4759-a7db-fb11f6255187 None None] MFA Rules not processed for user `89b765738e884fd3a1cbb2e1409eca7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:46.831205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9de3c07b-9813-4759-a7db-fb11f6255187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:46.838230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123ea10-4b6a-4c44-b7c2-9f05a48f3753 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:46.838736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123ea10-4b6a-4c44-b7c2-9f05a48f3753 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:46.839273 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123ea10-4b6a-4c44-b7c2-9f05a48f3753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:46.919939 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d123ea10-4b6a-4c44-b7c2-9f05a48f3753 None None] MFA Rules not processed for user `1a968944f4bc4c5b93f98d124b88fe01`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:46.921887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d123ea10-4b6a-4c44-b7c2-9f05a48f3753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:46.927626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:46.935655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 937/1895] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:46.950462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3c07b-9813-4759-a7db-fb11f6255187 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:46.956015 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9de3c07b-9813-4759-a7db-fb11f6255187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:47.047243 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 959/1896] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:14:47.060826 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3c07b-9813-4759-a7db-fb11f6255187 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:47.066381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2442a-0e1d-44c9-92de-c67b0c9b9d3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:47.066381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2442a-0e1d-44c9-92de-c67b0c9b9d3b service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:47.066381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2442a-0e1d-44c9-92de-c67b0c9b9d3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:47.066381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2442a-0e1d-44c9-92de-c67b0c9b9d3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:47.079076 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2442a-0e1d-44c9-92de-c67b0c9b9d3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:47.080556 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2442a-0e1d-44c9-92de-c67b0c9b9d3b service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:47.095031 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 938/1897] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:14:47.188465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123ea10-4b6a-4c44-b7c2-9f05a48f3753 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:47.191295 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d123ea10-4b6a-4c44-b7c2-9f05a48f3753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:47.207950 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123ea10-4b6a-4c44-b7c2-9f05a48f3753 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:47.213673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2aaa90-7726-4cce-b230-5ad70b0d66a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:47.214375 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2aaa90-7726-4cce-b230-5ad70b0d66a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:47.214742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2aaa90-7726-4cce-b230-5ad70b0d66a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:47.215103 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2aaa90-7726-4cce-b230-5ad70b0d66a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:47.228827 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2aaa90-7726-4cce-b230-5ad70b0d66a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:47.230998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2aaa90-7726-4cce-b230-5ad70b0d66a3 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:47.258102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 960/1898] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:14:51.026603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2442a-0e1d-44c9-92de-c67b0c9b9d3b service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:51.028111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-94e2442a-0e1d-44c9-92de-c67b0c9b9d3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:51.097743 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2442a-0e1d-44c9-92de-c67b0c9b9d3b service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:51.104105 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea5b0f-982d-4d32-9cd0-72b895c3becc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:51.104553 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea5b0f-982d-4d32-9cd0-72b895c3becc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:51.104861 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea5b0f-982d-4d32-9cd0-72b895c3becc admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:51.105130 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea5b0f-982d-4d32-9cd0-72b895c3becc admin admin] PATH_INFO: `/v3/users/87dfa73cede746dea9a677b125d498f8` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:51.106261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea5b0f-982d-4d32-9cd0-72b895c3becc admin admin] RBAC: Authorizing `identity:delete_user(user_id=87dfa73cede746dea9a677b125d498f8)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:51.108272 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea5b0f-982d-4d32-9cd0-72b895c3becc admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:51.373038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-faea5b0f-982d-4d32-9cd0-72b895c3becc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87dfa73cede746dea9a677b125d498f8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:51.373777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-faea5b0f-982d-4d32-9cd0-72b895c3becc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87dfa73cede746dea9a677b125d498f8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:51.386266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-faea5b0f-982d-4d32-9cd0-72b895c3becc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87dfa73cede746dea9a677b125d498f8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:51.489983 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-faea5b0f-982d-4d32-9cd0-72b895c3becc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87dfa73cede746dea9a677b125d498f8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:51.490628 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-faea5b0f-982d-4d32-9cd0-72b895c3becc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87dfa73cede746dea9a677b125d498f8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:51.504496 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 939/1899] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:51 2022] DELETE /identity/v3/users/87dfa73cede746dea9a677b125d498f8 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:14:51.508598 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0a64f-7cb5-4597-ba1e-78a08b7dc885 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:51.509414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0a64f-7cb5-4597-ba1e-78a08b7dc885 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:51.509414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0a64f-7cb5-4597-ba1e-78a08b7dc885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:51.529915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-67d0a64f-7cb5-4597-ba1e-78a08b7dc885 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:51.531275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-67d0a64f-7cb5-4597-ba1e-78a08b7dc885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:51.595004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 961/1900] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:51.606382 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea5b0f-982d-4d32-9cd0-72b895c3becc admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:51.607870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-faea5b0f-982d-4d32-9cd0-72b895c3becc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:51.635532 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:51.672596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea5b0f-982d-4d32-9cd0-72b895c3becc admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:51.676240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b3d0e-cab3-4a40-8657-9b667c724e0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:51.676900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b3d0e-cab3-4a40-8657-9b667c724e0f service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:51.677279 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b3d0e-cab3-4a40-8657-9b667c724e0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:51.677544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b3d0e-cab3-4a40-8657-9b667c724e0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:51.686986 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b3d0e-cab3-4a40-8657-9b667c724e0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:51.688419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b3d0e-cab3-4a40-8657-9b667c724e0f service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:51.698884 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 940/1901] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:14:52.269258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0a64f-7cb5-4597-ba1e-78a08b7dc885 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:52.272654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-67d0a64f-7cb5-4597-ba1e-78a08b7dc885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:52.279044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:52.284603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0a64f-7cb5-4597-ba1e-78a08b7dc885 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:52.289349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d6d58-c517-44f5-a846-6c47ad8348e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:52.289987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d6d58-c517-44f5-a846-6c47ad8348e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:52.290266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d6d58-c517-44f5-a846-6c47ad8348e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:52.290536 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d6d58-c517-44f5-a846-6c47ad8348e0 admin admin] PATH_INFO: `/v3/projects/aa257667f4b54140ade2962bf1b8a5c4` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:52.291870 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d6d58-c517-44f5-a846-6c47ad8348e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa257667f4b54140ade2962bf1b8a5c4)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:52.293804 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d6d58-c517-44f5-a846-6c47ad8348e0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:52.404574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-228d6d58-c517-44f5-a846-6c47ad8348e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa257667f4b54140ade2962bf1b8a5c4', 'request_id': 'req-228d6d58-c517-44f5-a846-6c47ad8348e0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:52.410754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-228d6d58-c517-44f5-a846-6c47ad8348e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa257667f4b54140ade2962bf1b8a5c4', 'request_id': 'req-228d6d58-c517-44f5-a846-6c47ad8348e0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:52.472589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-228d6d58-c517-44f5-a846-6c47ad8348e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa257667f4b54140ade2962bf1b8a5c4', 'request_id': 'req-228d6d58-c517-44f5-a846-6c47ad8348e0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:52.483778 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-228d6d58-c517-44f5-a846-6c47ad8348e0 admin admin] The token cache is being invalidate because project aa257667f4b54140ade2962bf1b8a5c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:52.484068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-228d6d58-c517-44f5-a846-6c47ad8348e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:52.484701 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-228d6d58-c517-44f5-a846-6c47ad8348e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:52.486210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 962/1902] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:52 2022] DELETE /identity/v3/projects/aa257667f4b54140ade2962bf1b8a5c4 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:52.490446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b3d0e-cab3-4a40-8657-9b667c724e0f service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:52.500612 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b3d0e-cab3-4a40-8657-9b667c724e0f service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:52.556045 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ce15e-3fa2-446a-aab7-30b79c5fb403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:52.556598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ce15e-3fa2-446a-aab7-30b79c5fb403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:52.557907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ce15e-3fa2-446a-aab7-30b79c5fb403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:52.558187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ce15e-3fa2-446a-aab7-30b79c5fb403 admin admin] PATH_INFO: `/v3/users/eda467a493484b32b9e42d0597aaa2a3` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:52.559353 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ce15e-3fa2-446a-aab7-30b79c5fb403 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda467a493484b32b9e42d0597aaa2a3)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:52.561315 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ce15e-3fa2-446a-aab7-30b79c5fb403 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:52.733359 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b52ce15e-3fa2-446a-aab7-30b79c5fb403 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda467a493484b32b9e42d0597aaa2a3'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:52.734132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b52ce15e-3fa2-446a-aab7-30b79c5fb403 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda467a493484b32b9e42d0597aaa2a3'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:52.747108 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b52ce15e-3fa2-446a-aab7-30b79c5fb403 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda467a493484b32b9e42d0597aaa2a3'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:52.804090 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b52ce15e-3fa2-446a-aab7-30b79c5fb403 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda467a493484b32b9e42d0597aaa2a3'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:52.805162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b52ce15e-3fa2-446a-aab7-30b79c5fb403 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda467a493484b32b9e42d0597aaa2a3'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:52.823773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 941/1903] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:14:52 2022] DELETE /identity/v3/users/eda467a493484b32b9e42d0597aaa2a3 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:53.268296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d6d58-c517-44f5-a846-6c47ad8348e0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:53.269854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-228d6d58-c517-44f5-a846-6c47ad8348e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:53.283205 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d6d58-c517-44f5-a846-6c47ad8348e0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:53.340740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3ce46-5ad4-4356-814b-7ea2ddff1d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:53.341365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3ce46-5ad4-4356-814b-7ea2ddff1d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:53.341639 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3ce46-5ad4-4356-814b-7ea2ddff1d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:53.341914 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3ce46-5ad4-4356-814b-7ea2ddff1d78 admin admin] PATH_INFO: `/v3/projects/a7df74a97ef340fa99fe62b57628d529` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:53.343051 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3ce46-5ad4-4356-814b-7ea2ddff1d78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7df74a97ef340fa99fe62b57628d529)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:53.344956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3ce46-5ad4-4356-814b-7ea2ddff1d78 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:53.468704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3de3ce46-5ad4-4356-814b-7ea2ddff1d78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7df74a97ef340fa99fe62b57628d529', 'request_id': 'req-3de3ce46-5ad4-4356-814b-7ea2ddff1d78'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:53.481352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3de3ce46-5ad4-4356-814b-7ea2ddff1d78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7df74a97ef340fa99fe62b57628d529', 'request_id': 'req-3de3ce46-5ad4-4356-814b-7ea2ddff1d78'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:53.535404 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3de3ce46-5ad4-4356-814b-7ea2ddff1d78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7df74a97ef340fa99fe62b57628d529', 'request_id': 'req-3de3ce46-5ad4-4356-814b-7ea2ddff1d78'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:53.546044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3de3ce46-5ad4-4356-814b-7ea2ddff1d78 admin admin] The token cache is being invalidate because project a7df74a97ef340fa99fe62b57628d529 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:14:53.546330 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3de3ce46-5ad4-4356-814b-7ea2ddff1d78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:53.546978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3de3ce46-5ad4-4356-814b-7ea2ddff1d78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:14:53.548489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 963/1904] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:14:53 2022] DELETE /identity/v3/projects/a7df74a97ef340fa99fe62b57628d529 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:53.553325 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7813d8-84c5-4fc9-8d50-8610a2f850a9 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:53.553772 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7813d8-84c5-4fc9-8d50-8610a2f850a9 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:53.553875 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7813d8-84c5-4fc9-8d50-8610a2f850a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:53.574914 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-3a7813d8-84c5-4fc9-8d50-8610a2f850a9 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:53.576143 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3a7813d8-84c5-4fc9-8d50-8610a2f850a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:53.615775 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:53.636312 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 942/1905] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:53.644083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02396f-bac9-4b35-9858-eb10b3b380ba None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:53.644363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02396f-bac9-4b35-9858-eb10b3b380ba None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:53.644633 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02396f-bac9-4b35-9858-eb10b3b380ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:53.665401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8f02396f-bac9-4b35-9858-eb10b3b380ba None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:53.667074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8f02396f-bac9-4b35-9858-eb10b3b380ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:53.675089 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 964/1906] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:53.679162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7813d8-84c5-4fc9-8d50-8610a2f850a9 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:53.689174 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7813d8-84c5-4fc9-8d50-8610a2f850a9 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:53.693669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e874c93-db52-4a75-ae1f-f44b80dcde56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:53.694263 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e874c93-db52-4a75-ae1f-f44b80dcde56 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:53.694562 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e874c93-db52-4a75-ae1f-f44b80dcde56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:53.694834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e874c93-db52-4a75-ae1f-f44b80dcde56 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:53.695512 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e874c93-db52-4a75-ae1f-f44b80dcde56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:14:53.695841 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e874c93-db52-4a75-ae1f-f44b80dcde56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:53.697564 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e874c93-db52-4a75-ae1f-f44b80dcde56 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:53.708895 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 943/1907] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:14:53.713306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02396f-bac9-4b35-9858-eb10b3b380ba None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:53.722538 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02396f-bac9-4b35-9858-eb10b3b380ba None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:53.727306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8494c1-6714-457e-a67f-4db62c6a8a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:53.727927 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8494c1-6714-457e-a67f-4db62c6a8a9c admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:53.728204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8494c1-6714-457e-a67f-4db62c6a8a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:53.728996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8494c1-6714-457e-a67f-4db62c6a8a9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:53.729763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8494c1-6714-457e-a67f-4db62c6a8a9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:53.731734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8494c1-6714-457e-a67f-4db62c6a8a9c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:53.737094 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:53.845213 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 965/1908] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:53 2022] POST /identity/v3/projects => generated 407 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:53.849486 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e874c93-db52-4a75-ae1f-f44b80dcde56 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:53.861019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e874c93-db52-4a75-ae1f-f44b80dcde56 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:53.927555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd1f84-0fba-4d2a-8efb-afa8463a0246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:53.929533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd1f84-0fba-4d2a-8efb-afa8463a0246 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:53.929533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd1f84-0fba-4d2a-8efb-afa8463a0246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:53.929533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd1f84-0fba-4d2a-8efb-afa8463a0246 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:53.929533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd1f84-0fba-4d2a-8efb-afa8463a0246 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:53.931327 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd1f84-0fba-4d2a-8efb-afa8463a0246 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:54.055994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 944/1909] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:53 2022] POST /identity/v3/users => generated 304 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:54.060860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8494c1-6714-457e-a67f-4db62c6a8a9c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:54.070958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8494c1-6714-457e-a67f-4db62c6a8a9c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:54.076155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ed5ef-14d3-4d7b-abb5-4a2c1f09bffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:54.076794 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ed5ef-14d3-4d7b-abb5-4a2c1f09bffd admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:54.077070 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ed5ef-14d3-4d7b-abb5-4a2c1f09bffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:54.077343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ed5ef-14d3-4d7b-abb5-4a2c1f09bffd admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:54.078019 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ed5ef-14d3-4d7b-abb5-4a2c1f09bffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:54.079648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ed5ef-14d3-4d7b-abb5-4a2c1f09bffd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:54.089711 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 966/1910] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:54.093828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd1f84-0fba-4d2a-8efb-afa8463a0246 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:54.104035 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd1f84-0fba-4d2a-8efb-afa8463a0246 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:54.108911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214370d2-979b-441c-998a-eef466b6e8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:54.109510 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214370d2-979b-441c-998a-eef466b6e8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:54.109816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214370d2-979b-441c-998a-eef466b6e8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:54.110089 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214370d2-979b-441c-998a-eef466b6e8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:54.110750 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214370d2-979b-441c-998a-eef466b6e8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:54.112396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214370d2-979b-441c-998a-eef466b6e8e6 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:54.123672 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 945/1911] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:54.127811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ed5ef-14d3-4d7b-abb5-4a2c1f09bffd admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:54.138742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ed5ef-14d3-4d7b-abb5-4a2c1f09bffd admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:54.143388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ca531-eba6-4d18-b225-cf58ae327363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:54.144046 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ca531-eba6-4d18-b225-cf58ae327363 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:54.144324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ca531-eba6-4d18-b225-cf58ae327363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:54.144600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ca531-eba6-4d18-b225-cf58ae327363 admin admin] PATH_INFO: `/v3/projects/ab20a4e3a1a64aa785c20a51f452bc73/users/9bbef2ca698f4ffeab82ba6521bbaf0b/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:54.160106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ca531-eba6-4d18-b225-cf58ae327363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab20a4e3a1a64aa785c20a51f452bc73, user_id=9bbef2ca698f4ffeab82ba6521bbaf0b, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:54.162904 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ca531-eba6-4d18-b225-cf58ae327363 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:54.267204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 967/1912] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:54 2022] PUT /identity/v3/projects/ab20a4e3a1a64aa785c20a51f452bc73/users/9bbef2ca698f4ffeab82ba6521bbaf0b/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:54.278447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a1b8e-aff4-4df4-880c-b6f6ce43be47 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:54.278833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a1b8e-aff4-4df4-880c-b6f6ce43be47 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:54.279136 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a1b8e-aff4-4df4-880c-b6f6ce43be47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:54.309085 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-ef6a1b8e-aff4-4df4-880c-b6f6ce43be47 None None] MFA Rules not processed for user `9bbef2ca698f4ffeab82ba6521bbaf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:54.310297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-ef6a1b8e-aff4-4df4-880c-b6f6ce43be47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:54.374717 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 946/1913] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:54 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:54.379580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ca531-eba6-4d18-b225-cf58ae327363 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:54.441331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ca531-eba6-4d18-b225-cf58ae327363 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:54.445971 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dec75f-8385-40d3-ac74-68c65860de4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:54.446567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dec75f-8385-40d3-ac74-68c65860de4c admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:54.446843 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dec75f-8385-40d3-ac74-68c65860de4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:54.447143 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dec75f-8385-40d3-ac74-68c65860de4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:54.448857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dec75f-8385-40d3-ac74-68c65860de4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:54.450812 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dec75f-8385-40d3-ac74-68c65860de4c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:54.518962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 968/1914] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:14:54 2022] POST /identity/v3/projects => generated 405 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:54.522894 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a1b8e-aff4-4df4-880c-b6f6ce43be47 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:54.534151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a1b8e-aff4-4df4-880c-b6f6ce43be47 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:54.571834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:54.590883 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a89a3-ff1b-46b2-a2bc-efcbc5a3678b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:54.591665 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a89a3-ff1b-46b2-a2bc-efcbc5a3678b admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:54.591939 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a89a3-ff1b-46b2-a2bc-efcbc5a3678b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:54.592208 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a89a3-ff1b-46b2-a2bc-efcbc5a3678b admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:54.592839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a89a3-ff1b-46b2-a2bc-efcbc5a3678b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:54.594836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a89a3-ff1b-46b2-a2bc-efcbc5a3678b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:54.661561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 947/1915] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:14:54 2022] POST /identity/v3/users => generated 309 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:14:54.665550 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dec75f-8385-40d3-ac74-68c65860de4c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:54.675901 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dec75f-8385-40d3-ac74-68c65860de4c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:54.680488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244dfeca-a063-45ee-9e87-8ca55171b9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:54.681083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244dfeca-a063-45ee-9e87-8ca55171b9da admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:54.681357 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244dfeca-a063-45ee-9e87-8ca55171b9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:54.682125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244dfeca-a063-45ee-9e87-8ca55171b9da admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:54.682782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244dfeca-a063-45ee-9e87-8ca55171b9da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:54.684397 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244dfeca-a063-45ee-9e87-8ca55171b9da admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:54.695036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 969/1916] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:54.705239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a89a3-ff1b-46b2-a2bc-efcbc5a3678b admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:54.715861 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a89a3-ff1b-46b2-a2bc-efcbc5a3678b admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:54.720675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee36fe0-2c53-42ec-9bfa-a7ed5b32f16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:54.721331 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee36fe0-2c53-42ec-9bfa-a7ed5b32f16d admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:54.721608 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee36fe0-2c53-42ec-9bfa-a7ed5b32f16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:54.721877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee36fe0-2c53-42ec-9bfa-a7ed5b32f16d admin admin] PATH_INFO: `/v3/projects/5cf299028032452bbe88e7cbf5c8879a/users/e79bebf9bf524d1da6ad69326ffb78dc/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:54.736471 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee36fe0-2c53-42ec-9bfa-a7ed5b32f16d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cf299028032452bbe88e7cbf5c8879a, user_id=e79bebf9bf524d1da6ad69326ffb78dc, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:54.739178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee36fe0-2c53-42ec-9bfa-a7ed5b32f16d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:54.803784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 948/1917] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:54 2022] PUT /identity/v3/projects/5cf299028032452bbe88e7cbf5c8879a/users/e79bebf9bf524d1da6ad69326ffb78dc/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:54.808240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244dfeca-a063-45ee-9e87-8ca55171b9da admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:54.818206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244dfeca-a063-45ee-9e87-8ca55171b9da admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:54.875199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751de170-8de9-4846-922d-e3bd132feb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:54.875800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751de170-8de9-4846-922d-e3bd132feb6e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:54.876071 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751de170-8de9-4846-922d-e3bd132feb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:54.876337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751de170-8de9-4846-922d-e3bd132feb6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:54.876967 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751de170-8de9-4846-922d-e3bd132feb6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:54.878801 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751de170-8de9-4846-922d-e3bd132feb6e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:54.889399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 970/1918] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:14:54.893489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee36fe0-2c53-42ec-9bfa-a7ed5b32f16d admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:54.903934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee36fe0-2c53-42ec-9bfa-a7ed5b32f16d admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:54.908548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b8da8-55dc-4ff6-b983-7e86e1d97e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:54.909196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b8da8-55dc-4ff6-b983-7e86e1d97e36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:54.909470 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b8da8-55dc-4ff6-b983-7e86e1d97e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:54.909741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b8da8-55dc-4ff6-b983-7e86e1d97e36 admin admin] PATH_INFO: `/v3/projects/5cf299028032452bbe88e7cbf5c8879a/users/e79bebf9bf524d1da6ad69326ffb78dc/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:54.911695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b8da8-55dc-4ff6-b983-7e86e1d97e36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cf299028032452bbe88e7cbf5c8879a, user_id=e79bebf9bf524d1da6ad69326ffb78dc, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:54.914325 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b8da8-55dc-4ff6-b983-7e86e1d97e36 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:54.925420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 949/1919] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:14:54 2022] PUT /identity/v3/projects/5cf299028032452bbe88e7cbf5c8879a/users/e79bebf9bf524d1da6ad69326ffb78dc/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:14:54.936326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b5abb-3a99-4567-ab70-7c418538590f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:54.936607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b5abb-3a99-4567-ab70-7c418538590f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:54.936875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b5abb-3a99-4567-ab70-7c418538590f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:54.965661 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d44b5abb-3a99-4567-ab70-7c418538590f None None] MFA Rules not processed for user `e79bebf9bf524d1da6ad69326ffb78dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:14:54.966903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d44b5abb-3a99-4567-ab70-7c418538590f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:54.983872 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:14:55.036327 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 971/1920] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:14:54 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:14:55.048063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b8da8-55dc-4ff6-b983-7e86e1d97e36 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:55.049874 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-2e2b8da8-55dc-4ff6-b983-7e86e1d97e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:14:55.114552 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b8da8-55dc-4ff6-b983-7e86e1d97e36 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:55.118271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c57d81-532e-405f-be7d-6ac1837d0c29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:55.119131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c57d81-532e-405f-be7d-6ac1837d0c29 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:55.119919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c57d81-532e-405f-be7d-6ac1837d0c29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:55.120183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c57d81-532e-405f-be7d-6ac1837d0c29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:55.130005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c57d81-532e-405f-be7d-6ac1837d0c29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:55.131464 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c57d81-532e-405f-be7d-6ac1837d0c29 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:55.141970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 950/1921] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:55 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:14:55.465000 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b5abb-3a99-4567-ab70-7c418538590f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:14:55.474699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b5abb-3a99-4567-ab70-7c418538590f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:14:55.478336 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b305a1-696f-43a3-a097-8d9334e35639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:14:55.479703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b305a1-696f-43a3-a097-8d9334e35639 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:14:55.479980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b305a1-696f-43a3-a097-8d9334e35639 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:14:55.480250 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b305a1-696f-43a3-a097-8d9334e35639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:14:55.539993 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b305a1-696f-43a3-a097-8d9334e35639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:14:55.542036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b305a1-696f-43a3-a097-8d9334e35639 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:14:55.551079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 972/1922] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:14:55 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:05.053483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c57d81-532e-405f-be7d-6ac1837d0c29 service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:05.055198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-54c57d81-532e-405f-be7d-6ac1837d0c29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:05.126350 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c57d81-532e-405f-be7d-6ac1837d0c29 service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:05.131422 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48589d1a-9b83-4696-81fe-5bff38fbd81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:05.132058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48589d1a-9b83-4696-81fe-5bff38fbd81a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:05.132336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48589d1a-9b83-4696-81fe-5bff38fbd81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:05.132611 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48589d1a-9b83-4696-81fe-5bff38fbd81a admin admin] PATH_INFO: `/v3/users/89b765738e884fd3a1cbb2e1409eca7a` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:05.133735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48589d1a-9b83-4696-81fe-5bff38fbd81a admin admin] RBAC: Authorizing `identity:delete_user(user_id=89b765738e884fd3a1cbb2e1409eca7a)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:05.135727 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48589d1a-9b83-4696-81fe-5bff38fbd81a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:05.288914 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:05.299244 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-48589d1a-9b83-4696-81fe-5bff38fbd81a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89b765738e884fd3a1cbb2e1409eca7a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:05.299918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-48589d1a-9b83-4696-81fe-5bff38fbd81a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89b765738e884fd3a1cbb2e1409eca7a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:05.319264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-48589d1a-9b83-4696-81fe-5bff38fbd81a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89b765738e884fd3a1cbb2e1409eca7a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:05.369125 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-48589d1a-9b83-4696-81fe-5bff38fbd81a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89b765738e884fd3a1cbb2e1409eca7a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:05.370163 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-48589d1a-9b83-4696-81fe-5bff38fbd81a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89b765738e884fd3a1cbb2e1409eca7a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:05.387275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 951/1923] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:05 2022] DELETE /identity/v3/users/89b765738e884fd3a1cbb2e1409eca7a => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:05.391465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceda9d-6ce0-42d1-8cd3-a25ca466790e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:05.391985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceda9d-6ce0-42d1-8cd3-a25ca466790e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:05.392329 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceda9d-6ce0-42d1-8cd3-a25ca466790e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:05.414273 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7bceda9d-6ce0-42d1-8cd3-a25ca466790e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:05.416001 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7bceda9d-6ce0-42d1-8cd3-a25ca466790e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:05.478490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 973/1924] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:05.488632 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48589d1a-9b83-4696-81fe-5bff38fbd81a admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:05.490568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-48589d1a-9b83-4696-81fe-5bff38fbd81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:05.551222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48589d1a-9b83-4696-81fe-5bff38fbd81a admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:05.554583 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a695ed0-66ba-439d-bde3-20e29f7380e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:05.555206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a695ed0-66ba-439d-bde3-20e29f7380e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:05.555483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a695ed0-66ba-439d-bde3-20e29f7380e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:05.555750 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a695ed0-66ba-439d-bde3-20e29f7380e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:05.564631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a695ed0-66ba-439d-bde3-20e29f7380e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:05.566059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a695ed0-66ba-439d-bde3-20e29f7380e5 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:05.576099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 952/1925] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:06.039592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bceda9d-6ce0-42d1-8cd3-a25ca466790e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:06.041440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7bceda9d-6ce0-42d1-8cd3-a25ca466790e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:06.056288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bceda9d-6ce0-42d1-8cd3-a25ca466790e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:06.062674 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c01d74-ae4f-4127-97e6-7517b178b5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:06.063403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c01d74-ae4f-4127-97e6-7517b178b5d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:06.063620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c01d74-ae4f-4127-97e6-7517b178b5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:06.072447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c01d74-ae4f-4127-97e6-7517b178b5d1 admin admin] PATH_INFO: `/v3/projects/23ffa92fc476474bb1f95c75bf202cce` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:06.072447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c01d74-ae4f-4127-97e6-7517b178b5d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ffa92fc476474bb1f95c75bf202cce)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:06.072447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c01d74-ae4f-4127-97e6-7517b178b5d1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:06.164898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:06.170930 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-16c01d74-ae4f-4127-97e6-7517b178b5d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ffa92fc476474bb1f95c75bf202cce', 'request_id': 'req-16c01d74-ae4f-4127-97e6-7517b178b5d1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:06.177165 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-16c01d74-ae4f-4127-97e6-7517b178b5d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ffa92fc476474bb1f95c75bf202cce', 'request_id': 'req-16c01d74-ae4f-4127-97e6-7517b178b5d1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:06.226120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-16c01d74-ae4f-4127-97e6-7517b178b5d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ffa92fc476474bb1f95c75bf202cce', 'request_id': 'req-16c01d74-ae4f-4127-97e6-7517b178b5d1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:06.236879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-16c01d74-ae4f-4127-97e6-7517b178b5d1 admin admin] The token cache is being invalidate because project 23ffa92fc476474bb1f95c75bf202cce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:06.237341 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-16c01d74-ae4f-4127-97e6-7517b178b5d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:06.238438 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-16c01d74-ae4f-4127-97e6-7517b178b5d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:06.239999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 974/1926] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:06 2022] DELETE /identity/v3/projects/23ffa92fc476474bb1f95c75bf202cce => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:06.244982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3043a5-8149-4959-b57f-7487fb2ed757 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:06.245462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3043a5-8149-4959-b57f-7487fb2ed757 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:06.245587 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3043a5-8149-4959-b57f-7487fb2ed757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:06.266115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-cd3043a5-8149-4959-b57f-7487fb2ed757 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:06.267330 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-cd3043a5-8149-4959-b57f-7487fb2ed757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:06.327211 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 953/1927] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:06.334876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9fbd9-e577-4b60-8b67-21bc02c86715 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:06.335293 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9fbd9-e577-4b60-8b67-21bc02c86715 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:06.335497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9fbd9-e577-4b60-8b67-21bc02c86715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:06.357983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-30f9fbd9-e577-4b60-8b67-21bc02c86715 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:06.359252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-30f9fbd9-e577-4b60-8b67-21bc02c86715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:06.367782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 975/1928] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:06.373286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3043a5-8149-4959-b57f-7487fb2ed757 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:06.386194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3043a5-8149-4959-b57f-7487fb2ed757 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:06.391285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2c70c-3f85-4729-8666-ad86731cbdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:06.391898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2c70c-3f85-4729-8666-ad86731cbdf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:06.395582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2c70c-3f85-4729-8666-ad86731cbdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:06.395582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2c70c-3f85-4729-8666-ad86731cbdf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:06.395582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd2c70c-3f85-4729-8666-ad86731cbdf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:15:06.395582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd2c70c-3f85-4729-8666-ad86731cbdf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:06.395582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd2c70c-3f85-4729-8666-ad86731cbdf9 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:06.411149 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 954/1929] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:15:06.412133 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9fbd9-e577-4b60-8b67-21bc02c86715 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:06.422334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9fbd9-e577-4b60-8b67-21bc02c86715 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:06.427058 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428509d-18a5-44e4-90f8-6d749b5024c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:06.427642 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428509d-18a5-44e4-90f8-6d749b5024c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:06.427913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428509d-18a5-44e4-90f8-6d749b5024c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:06.428177 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428509d-18a5-44e4-90f8-6d749b5024c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:06.428813 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4428509d-18a5-44e4-90f8-6d749b5024c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:06.430734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4428509d-18a5-44e4-90f8-6d749b5024c9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:06.501328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 976/1930] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:06 2022] POST /identity/v3/projects => generated 359 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:15:06.505731 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2c70c-3f85-4729-8666-ad86731cbdf9 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:06.516448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2c70c-3f85-4729-8666-ad86731cbdf9 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:06.556387 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:06.571165 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa43d4e-465a-49df-bd32-d8f406ad6ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:06.571765 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa43d4e-465a-49df-bd32-d8f406ad6ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:06.572036 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa43d4e-465a-49df-bd32-d8f406ad6ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:06.572304 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa43d4e-465a-49df-bd32-d8f406ad6ce1 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:06.572961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa43d4e-465a-49df-bd32-d8f406ad6ce1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:06.575594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa43d4e-465a-49df-bd32-d8f406ad6ce1 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:06.654754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 955/1931] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:06 2022] POST /identity/v3/users => generated 280 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:06.658549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428509d-18a5-44e4-90f8-6d749b5024c9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:06.669227 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428509d-18a5-44e4-90f8-6d749b5024c9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:06.673925 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546093ee-cf8f-4472-b432-7e668777fd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:06.674538 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546093ee-cf8f-4472-b432-7e668777fd48 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:06.674816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546093ee-cf8f-4472-b432-7e668777fd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:06.675112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546093ee-cf8f-4472-b432-7e668777fd48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:06.675789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546093ee-cf8f-4472-b432-7e668777fd48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:06.677579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546093ee-cf8f-4472-b432-7e668777fd48 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:06.688532 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 977/1932] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:06.692641 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa43d4e-465a-49df-bd32-d8f406ad6ce1 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:06.703726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa43d4e-465a-49df-bd32-d8f406ad6ce1 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:06.710711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025b152-dd16-49ac-8a33-50e9c2636a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:06.713614 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025b152-dd16-49ac-8a33-50e9c2636a9b admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:06.713933 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025b152-dd16-49ac-8a33-50e9c2636a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:06.714810 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025b152-dd16-49ac-8a33-50e9c2636a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:06.715733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025b152-dd16-49ac-8a33-50e9c2636a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:06.717469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025b152-dd16-49ac-8a33-50e9c2636a9b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:06.728384 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 956/1933] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:06.732777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546093ee-cf8f-4472-b432-7e668777fd48 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:06.742787 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546093ee-cf8f-4472-b432-7e668777fd48 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:06.748284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a632a5b-41bc-4694-b635-a5994cb81293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:06.748958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a632a5b-41bc-4694-b635-a5994cb81293 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:06.749238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a632a5b-41bc-4694-b635-a5994cb81293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:06.749510 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a632a5b-41bc-4694-b635-a5994cb81293 admin admin] PATH_INFO: `/v3/projects/649ed4c8729843eda9ad489ff39e76f7/users/a0bfd02301f047a28716f02892da70a8/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:06.764860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a632a5b-41bc-4694-b635-a5994cb81293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=649ed4c8729843eda9ad489ff39e76f7, user_id=a0bfd02301f047a28716f02892da70a8, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:06.767570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a632a5b-41bc-4694-b635-a5994cb81293 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:06.850051 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 978/1934] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:06 2022] PUT /identity/v3/projects/649ed4c8729843eda9ad489ff39e76f7/users/a0bfd02301f047a28716f02892da70a8/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:06.860828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a89bcc-8298-4223-af93-5ff91c5e47cc None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:06.861206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a89bcc-8298-4223-af93-5ff91c5e47cc None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:06.861552 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a89bcc-8298-4223-af93-5ff91c5e47cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:06.890784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-67a89bcc-8298-4223-af93-5ff91c5e47cc None None] MFA Rules not processed for user `a0bfd02301f047a28716f02892da70a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:06.892146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-67a89bcc-8298-4223-af93-5ff91c5e47cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:06.961495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 957/1935] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:06 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:07.003004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a632a5b-41bc-4694-b635-a5994cb81293 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:07.005222 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4a632a5b-41bc-4694-b635-a5994cb81293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:07.098515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a632a5b-41bc-4694-b635-a5994cb81293 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:07.102271 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34da6a-4bde-4167-a579-5b89d6bd114b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:07.103536 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34da6a-4bde-4167-a579-5b89d6bd114b service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:07.103536 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34da6a-4bde-4167-a579-5b89d6bd114b service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:07.103536 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34da6a-4bde-4167-a579-5b89d6bd114b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:07.114641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34da6a-4bde-4167-a579-5b89d6bd114b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:07.116489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34da6a-4bde-4167-a579-5b89d6bd114b service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:07.128420 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 979/1936] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:07 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:07.146449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a89bcc-8298-4223-af93-5ff91c5e47cc None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:07.223790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a89bcc-8298-4223-af93-5ff91c5e47cc None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:07.229109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6423fe-03ee-4178-a417-3ae8555a85af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:07.230497 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6423fe-03ee-4178-a417-3ae8555a85af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:07.230979 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6423fe-03ee-4178-a417-3ae8555a85af admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:07.231088 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6423fe-03ee-4178-a417-3ae8555a85af admin admin] PATH_INFO: `/v3/users/a0bfd02301f047a28716f02892da70a8` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:07.232244 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6423fe-03ee-4178-a417-3ae8555a85af admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0bfd02301f047a28716f02892da70a8)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:07.234200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6423fe-03ee-4178-a417-3ae8555a85af admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:07.419934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bc6423fe-03ee-4178-a417-3ae8555a85af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0bfd02301f047a28716f02892da70a8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:07.421004 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bc6423fe-03ee-4178-a417-3ae8555a85af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0bfd02301f047a28716f02892da70a8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:07.429685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:07.438254 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bc6423fe-03ee-4178-a417-3ae8555a85af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0bfd02301f047a28716f02892da70a8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:07.496029 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bc6423fe-03ee-4178-a417-3ae8555a85af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0bfd02301f047a28716f02892da70a8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:07.497154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bc6423fe-03ee-4178-a417-3ae8555a85af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0bfd02301f047a28716f02892da70a8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:07.513907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 958/1937] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:07 2022] DELETE /identity/v3/users/a0bfd02301f047a28716f02892da70a8 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:07.523823 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc71b9-11ac-4317-b220-6becee5a7b36 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:07.529025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc71b9-11ac-4317-b220-6becee5a7b36 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:07.529025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc71b9-11ac-4317-b220-6becee5a7b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:07.553556 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f6bc71b9-11ac-4317-b220-6becee5a7b36 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:07.554811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f6bc71b9-11ac-4317-b220-6becee5a7b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:07.622403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 980/1938] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:07.632500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6423fe-03ee-4178-a417-3ae8555a85af admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:07.633960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-bc6423fe-03ee-4178-a417-3ae8555a85af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:07.694129 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6423fe-03ee-4178-a417-3ae8555a85af admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:07.697856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f9021-97d8-4748-aad3-1d5de64aae51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:07.698985 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f9021-97d8-4748-aad3-1d5de64aae51 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:07.699337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f9021-97d8-4748-aad3-1d5de64aae51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:07.699714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f9021-97d8-4748-aad3-1d5de64aae51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:07.709408 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f9021-97d8-4748-aad3-1d5de64aae51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:07.710910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f9021-97d8-4748-aad3-1d5de64aae51 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:07.722492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 959/1939] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:08.165858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc71b9-11ac-4317-b220-6becee5a7b36 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:08.168495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f6bc71b9-11ac-4317-b220-6becee5a7b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:08.183077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc71b9-11ac-4317-b220-6becee5a7b36 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:08.188545 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed3f0e-906e-4282-bbb0-fa65658725f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:08.189194 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed3f0e-906e-4282-bbb0-fa65658725f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:08.189467 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed3f0e-906e-4282-bbb0-fa65658725f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:08.189734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed3f0e-906e-4282-bbb0-fa65658725f2 admin admin] PATH_INFO: `/v3/projects/649ed4c8729843eda9ad489ff39e76f7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:08.192390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed3f0e-906e-4282-bbb0-fa65658725f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=649ed4c8729843eda9ad489ff39e76f7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:08.194905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed3f0e-906e-4282-bbb0-fa65658725f2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:08.197732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:08.304068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6bed3f0e-906e-4282-bbb0-fa65658725f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '649ed4c8729843eda9ad489ff39e76f7', 'request_id': 'req-6bed3f0e-906e-4282-bbb0-fa65658725f2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:08.310440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6bed3f0e-906e-4282-bbb0-fa65658725f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '649ed4c8729843eda9ad489ff39e76f7', 'request_id': 'req-6bed3f0e-906e-4282-bbb0-fa65658725f2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:08.379971 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6bed3f0e-906e-4282-bbb0-fa65658725f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '649ed4c8729843eda9ad489ff39e76f7', 'request_id': 'req-6bed3f0e-906e-4282-bbb0-fa65658725f2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:08.392026 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6bed3f0e-906e-4282-bbb0-fa65658725f2 admin admin] The token cache is being invalidate because project 649ed4c8729843eda9ad489ff39e76f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:08.392310 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6bed3f0e-906e-4282-bbb0-fa65658725f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:08.392949 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6bed3f0e-906e-4282-bbb0-fa65658725f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:08.394488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 981/1940] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:08 2022] DELETE /identity/v3/projects/649ed4c8729843eda9ad489ff39e76f7 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:08.399345 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08f90f-490a-48c4-adb8-6c06218f2ac6 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:08.399630 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08f90f-490a-48c4-adb8-6c06218f2ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:08.399903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08f90f-490a-48c4-adb8-6c06218f2ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:08.420337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-1a08f90f-490a-48c4-adb8-6c06218f2ac6 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:08.421523 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1a08f90f-490a-48c4-adb8-6c06218f2ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:08.480024 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 960/1941] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:08.488828 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00928363-f580-4722-b89b-5a6dc771ce3e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:08.489112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00928363-f580-4722-b89b-5a6dc771ce3e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:08.489385 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00928363-f580-4722-b89b-5a6dc771ce3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:08.510558 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-00928363-f580-4722-b89b-5a6dc771ce3e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:08.512315 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-00928363-f580-4722-b89b-5a6dc771ce3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:08.520561 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 982/1942] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:08.524681 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08f90f-490a-48c4-adb8-6c06218f2ac6 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:08.535917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08f90f-490a-48c4-adb8-6c06218f2ac6 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:08.540755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af34a5-0a71-4ada-b121-53724b874183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:08.541361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af34a5-0a71-4ada-b121-53724b874183 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:08.541641 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af34a5-0a71-4ada-b121-53724b874183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:08.541962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af34a5-0a71-4ada-b121-53724b874183 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:08.542630 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af34a5-0a71-4ada-b121-53724b874183 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:15:08.542923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af34a5-0a71-4ada-b121-53724b874183 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:08.544562 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af34a5-0a71-4ada-b121-53724b874183 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:08.556793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 961/1943] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:15:08.561586 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00928363-f580-4722-b89b-5a6dc771ce3e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:08.571590 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00928363-f580-4722-b89b-5a6dc771ce3e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:08.577212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f721be1-5bf8-4fbe-b026-a988795345b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:08.577881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f721be1-5bf8-4fbe-b026-a988795345b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:08.578163 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f721be1-5bf8-4fbe-b026-a988795345b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:08.578436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f721be1-5bf8-4fbe-b026-a988795345b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:08.579197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f721be1-5bf8-4fbe-b026-a988795345b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:08.581095 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f721be1-5bf8-4fbe-b026-a988795345b4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:08.684045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 983/1944] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:08 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:08.696836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af34a5-0a71-4ada-b121-53724b874183 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:08.712388 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af34a5-0a71-4ada-b121-53724b874183 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:08.774183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a76b5-7b7c-45c2-bd3d-0edd2e5c3608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:08.774769 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a76b5-7b7c-45c2-bd3d-0edd2e5c3608 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:08.775197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a76b5-7b7c-45c2-bd3d-0edd2e5c3608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:08.775527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a76b5-7b7c-45c2-bd3d-0edd2e5c3608 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:08.776162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a76b5-7b7c-45c2-bd3d-0edd2e5c3608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:08.778225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a76b5-7b7c-45c2-bd3d-0edd2e5c3608 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:08.787777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:09.029456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 962/1945] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:08 2022] POST /identity/v3/users => generated 291 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:09.030751 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f721be1-5bf8-4fbe-b026-a988795345b4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:09.041271 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f721be1-5bf8-4fbe-b026-a988795345b4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:09.046142 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17da0a9-ea93-406a-a90a-35c3855b98d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:09.046738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17da0a9-ea93-406a-a90a-35c3855b98d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:09.047064 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17da0a9-ea93-406a-a90a-35c3855b98d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:09.047340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17da0a9-ea93-406a-a90a-35c3855b98d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:09.047987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17da0a9-ea93-406a-a90a-35c3855b98d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:09.049584 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17da0a9-ea93-406a-a90a-35c3855b98d7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:09.061377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 984/1946] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:09.065434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a76b5-7b7c-45c2-bd3d-0edd2e5c3608 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:09.076797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a76b5-7b7c-45c2-bd3d-0edd2e5c3608 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:09.081654 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c5c37-ff2d-4b5c-bf49-2b568dc51a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:09.082303 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c5c37-ff2d-4b5c-bf49-2b568dc51a2d admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:09.082585 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c5c37-ff2d-4b5c-bf49-2b568dc51a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:09.082857 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c5c37-ff2d-4b5c-bf49-2b568dc51a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:09.083540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c5c37-ff2d-4b5c-bf49-2b568dc51a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:09.085168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c5c37-ff2d-4b5c-bf49-2b568dc51a2d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:09.096359 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 963/1947] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:09.100453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17da0a9-ea93-406a-a90a-35c3855b98d7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:09.111246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17da0a9-ea93-406a-a90a-35c3855b98d7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:09.115746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23723bea-c445-40db-9782-8942a66e1cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:09.116376 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23723bea-c445-40db-9782-8942a66e1cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:09.116647 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23723bea-c445-40db-9782-8942a66e1cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:09.116912 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23723bea-c445-40db-9782-8942a66e1cd5 admin admin] PATH_INFO: `/v3/projects/e3df00e25d7d49a98f98abc484148f92/users/5d74c0b6835a483c81c10330c9ed8663/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:09.131411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23723bea-c445-40db-9782-8942a66e1cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3df00e25d7d49a98f98abc484148f92, user_id=5d74c0b6835a483c81c10330c9ed8663, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:09.134042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23723bea-c445-40db-9782-8942a66e1cd5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:09.215884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 985/1948] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:09 2022] PUT /identity/v3/projects/e3df00e25d7d49a98f98abc484148f92/users/5d74c0b6835a483c81c10330c9ed8663/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:09.226221 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4db0b8-cf83-416a-b2d2-8a86a8c86aff None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:09.226502 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4db0b8-cf83-416a-b2d2-8a86a8c86aff None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:09.226770 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4db0b8-cf83-416a-b2d2-8a86a8c86aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:09.256150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-ef4db0b8-cf83-416a-b2d2-8a86a8c86aff None None] MFA Rules not processed for user `5d74c0b6835a483c81c10330c9ed8663`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:09.257360 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-ef4db0b8-cf83-416a-b2d2-8a86a8c86aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:09.325904 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 964/1949] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:09.342250 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23723bea-c445-40db-9782-8942a66e1cd5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:09.344318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-23723bea-c445-40db-9782-8942a66e1cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:09.422217 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23723bea-c445-40db-9782-8942a66e1cd5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:09.425881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31851603-7ca5-43ec-83cc-b55a2560409b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:09.426489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31851603-7ca5-43ec-83cc-b55a2560409b service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:09.426768 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31851603-7ca5-43ec-83cc-b55a2560409b service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:09.427071 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31851603-7ca5-43ec-83cc-b55a2560409b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:09.436600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31851603-7ca5-43ec-83cc-b55a2560409b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:09.438045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31851603-7ca5-43ec-83cc-b55a2560409b service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:09.447129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 986/1950] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:15:13.096514 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4db0b8-cf83-416a-b2d2-8a86a8c86aff None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:13.098040 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-ef4db0b8-cf83-416a-b2d2-8a86a8c86aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:13.177861 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4db0b8-cf83-416a-b2d2-8a86a8c86aff None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:13.182761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cdee0f-5023-4a57-818c-0179a680ae5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:13.183438 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdee0f-5023-4a57-818c-0179a680ae5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:13.183716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdee0f-5023-4a57-818c-0179a680ae5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:13.183974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdee0f-5023-4a57-818c-0179a680ae5e admin admin] PATH_INFO: `/v3/users/9bbef2ca698f4ffeab82ba6521bbaf0b` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:13.185094 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cdee0f-5023-4a57-818c-0179a680ae5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bbef2ca698f4ffeab82ba6521bbaf0b)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:13.187034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cdee0f-5023-4a57-818c-0179a680ae5e admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:13.372111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-81cdee0f-5023-4a57-818c-0179a680ae5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bbef2ca698f4ffeab82ba6521bbaf0b'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:13.372824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-81cdee0f-5023-4a57-818c-0179a680ae5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bbef2ca698f4ffeab82ba6521bbaf0b'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:13.387636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-81cdee0f-5023-4a57-818c-0179a680ae5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bbef2ca698f4ffeab82ba6521bbaf0b'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:13.460625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-81cdee0f-5023-4a57-818c-0179a680ae5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bbef2ca698f4ffeab82ba6521bbaf0b'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:13.461172 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-81cdee0f-5023-4a57-818c-0179a680ae5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bbef2ca698f4ffeab82ba6521bbaf0b'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:13.462475 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:13.474502 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 965/1951] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:13 2022] DELETE /identity/v3/users/9bbef2ca698f4ffeab82ba6521bbaf0b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:13.479079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0826eb-cebd-4a7b-b1c9-639dd1a0454b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:13.480139 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0826eb-cebd-4a7b-b1c9-639dd1a0454b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:13.481003 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0826eb-cebd-4a7b-b1c9-639dd1a0454b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:13.502906 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9e0826eb-cebd-4a7b-b1c9-639dd1a0454b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:13.504367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9e0826eb-cebd-4a7b-b1c9-639dd1a0454b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:13.520250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cdee0f-5023-4a57-818c-0179a680ae5e admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:13.521636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-81cdee0f-5023-4a57-818c-0179a680ae5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:13.532628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:13.568279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 987/1952] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:13.579357 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0826eb-cebd-4a7b-b1c9-639dd1a0454b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:13.580985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9e0826eb-cebd-4a7b-b1c9-639dd1a0454b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:13.593229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cdee0f-5023-4a57-818c-0179a680ae5e admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:13.598063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e97c3-133a-4d45-aec9-91036f5a7f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:13.598686 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e97c3-133a-4d45-aec9-91036f5a7f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:13.599022 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e97c3-133a-4d45-aec9-91036f5a7f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:13.599243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e97c3-133a-4d45-aec9-91036f5a7f80 admin admin] PATH_INFO: `/v3/users/5d74c0b6835a483c81c10330c9ed8663` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:13.600379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e97c3-133a-4d45-aec9-91036f5a7f80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d74c0b6835a483c81c10330c9ed8663)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:13.602295 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e97c3-133a-4d45-aec9-91036f5a7f80 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:13.642805 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0826eb-cebd-4a7b-b1c9-639dd1a0454b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:13.652252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7ff77-56c5-4cef-bdfe-749ed9a9ce08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:13.652252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7ff77-56c5-4cef-bdfe-749ed9a9ce08 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:13.652252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7ff77-56c5-4cef-bdfe-749ed9a9ce08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:13.652252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7ff77-56c5-4cef-bdfe-749ed9a9ce08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:13.661734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7ff77-56c5-4cef-bdfe-749ed9a9ce08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:13.663343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7ff77-56c5-4cef-bdfe-749ed9a9ce08 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:13.674752 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 988/1953] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:15:13.766591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-907e97c3-133a-4d45-aec9-91036f5a7f80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d74c0b6835a483c81c10330c9ed8663'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:13.767406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-907e97c3-133a-4d45-aec9-91036f5a7f80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d74c0b6835a483c81c10330c9ed8663'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:13.780592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-907e97c3-133a-4d45-aec9-91036f5a7f80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d74c0b6835a483c81c10330c9ed8663'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:13.900725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-907e97c3-133a-4d45-aec9-91036f5a7f80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d74c0b6835a483c81c10330c9ed8663'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:13.901944 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-907e97c3-133a-4d45-aec9-91036f5a7f80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d74c0b6835a483c81c10330c9ed8663'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:13.916582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 966/1954] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:13 2022] DELETE /identity/v3/users/5d74c0b6835a483c81c10330c9ed8663 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:13.922212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df191f9a-19e3-447b-81ca-605162a376cb None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:13.922624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df191f9a-19e3-447b-81ca-605162a376cb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:13.923545 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df191f9a-19e3-447b-81ca-605162a376cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:13.945033 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-df191f9a-19e3-447b-81ca-605162a376cb None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:13.946319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-df191f9a-19e3-447b-81ca-605162a376cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:14.051104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 989/1955] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:14.061630 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e97c3-133a-4d45-aec9-91036f5a7f80 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:14.063225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-907e97c3-133a-4d45-aec9-91036f5a7f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:14.132939 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e97c3-133a-4d45-aec9-91036f5a7f80 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:14.137654 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28f245-3cc2-4c8b-b53c-50b6340aec24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:14.138264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28f245-3cc2-4c8b-b53c-50b6340aec24 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:14.138684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28f245-3cc2-4c8b-b53c-50b6340aec24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:14.138994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28f245-3cc2-4c8b-b53c-50b6340aec24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:14.159126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28f245-3cc2-4c8b-b53c-50b6340aec24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:14.160453 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28f245-3cc2-4c8b-b53c-50b6340aec24 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:14.172138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 967/1956] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:14.285460 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df191f9a-19e3-447b-81ca-605162a376cb None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:14.287866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-df191f9a-19e3-447b-81ca-605162a376cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:14.311089 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df191f9a-19e3-447b-81ca-605162a376cb None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:14.326662 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fffdb7-a96f-4028-b15c-01a3a94faba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:14.327387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fffdb7-a96f-4028-b15c-01a3a94faba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:14.327667 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fffdb7-a96f-4028-b15c-01a3a94faba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:14.327922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fffdb7-a96f-4028-b15c-01a3a94faba8 admin admin] PATH_INFO: `/v3/projects/ab20a4e3a1a64aa785c20a51f452bc73` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:14.329066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fffdb7-a96f-4028-b15c-01a3a94faba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab20a4e3a1a64aa785c20a51f452bc73)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:14.330999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fffdb7-a96f-4028-b15c-01a3a94faba8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:14.481235 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-03fffdb7-a96f-4028-b15c-01a3a94faba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab20a4e3a1a64aa785c20a51f452bc73', 'request_id': 'req-03fffdb7-a96f-4028-b15c-01a3a94faba8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:14.488669 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-03fffdb7-a96f-4028-b15c-01a3a94faba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab20a4e3a1a64aa785c20a51f452bc73', 'request_id': 'req-03fffdb7-a96f-4028-b15c-01a3a94faba8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:14.562259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-03fffdb7-a96f-4028-b15c-01a3a94faba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab20a4e3a1a64aa785c20a51f452bc73', 'request_id': 'req-03fffdb7-a96f-4028-b15c-01a3a94faba8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:14.564507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:14.570107 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-03fffdb7-a96f-4028-b15c-01a3a94faba8 admin admin] The token cache is being invalidate because project ab20a4e3a1a64aa785c20a51f452bc73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:14.570390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-03fffdb7-a96f-4028-b15c-01a3a94faba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:14.571071 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-03fffdb7-a96f-4028-b15c-01a3a94faba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:14.572562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 990/1957] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:14 2022] DELETE /identity/v3/projects/ab20a4e3a1a64aa785c20a51f452bc73 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:14.577194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28f245-3cc2-4c8b-b53c-50b6340aec24 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:14.587141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28f245-3cc2-4c8b-b53c-50b6340aec24 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:14.596069 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:14.643945 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438db982-3193-4f5f-a462-c3374943ee9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:14.644568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438db982-3193-4f5f-a462-c3374943ee9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:14.644853 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438db982-3193-4f5f-a462-c3374943ee9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:14.645126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438db982-3193-4f5f-a462-c3374943ee9f admin admin] PATH_INFO: `/v3/users/e79bebf9bf524d1da6ad69326ffb78dc` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:14.646247 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438db982-3193-4f5f-a462-c3374943ee9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e79bebf9bf524d1da6ad69326ffb78dc)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:14.648214 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438db982-3193-4f5f-a462-c3374943ee9f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:14.667856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fffdb7-a96f-4028-b15c-01a3a94faba8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:14.669549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-03fffdb7-a96f-4028-b15c-01a3a94faba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:14.686348 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fffdb7-a96f-4028-b15c-01a3a94faba8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:14.691949 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb9d47-a5e8-4737-ade7-d1a797eebd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:14.692473 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb9d47-a5e8-4737-ade7-d1a797eebd84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:14.693138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb9d47-a5e8-4737-ade7-d1a797eebd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:14.693486 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb9d47-a5e8-4737-ade7-d1a797eebd84 admin admin] PATH_INFO: `/v3/projects/e3df00e25d7d49a98f98abc484148f92` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:14.694559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb9d47-a5e8-4737-ade7-d1a797eebd84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3df00e25d7d49a98f98abc484148f92)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:14.696691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb9d47-a5e8-4737-ade7-d1a797eebd84 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:14.845997 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-438db982-3193-4f5f-a462-c3374943ee9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e79bebf9bf524d1da6ad69326ffb78dc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:14.846688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-438db982-3193-4f5f-a462-c3374943ee9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e79bebf9bf524d1da6ad69326ffb78dc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:14.846812 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e3fb9d47-a5e8-4737-ade7-d1a797eebd84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3df00e25d7d49a98f98abc484148f92', 'request_id': 'req-e3fb9d47-a5e8-4737-ade7-d1a797eebd84'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:14.853882 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e3fb9d47-a5e8-4737-ade7-d1a797eebd84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3df00e25d7d49a98f98abc484148f92', 'request_id': 'req-e3fb9d47-a5e8-4737-ade7-d1a797eebd84'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:14.859126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-438db982-3193-4f5f-a462-c3374943ee9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e79bebf9bf524d1da6ad69326ffb78dc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:14.906073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e3fb9d47-a5e8-4737-ade7-d1a797eebd84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3df00e25d7d49a98f98abc484148f92', 'request_id': 'req-e3fb9d47-a5e8-4737-ade7-d1a797eebd84'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:14.917316 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e3fb9d47-a5e8-4737-ade7-d1a797eebd84 admin admin] The token cache is being invalidate because project e3df00e25d7d49a98f98abc484148f92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:14.917630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e3fb9d47-a5e8-4737-ade7-d1a797eebd84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:14.918310 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e3fb9d47-a5e8-4737-ade7-d1a797eebd84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:14.919892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 991/1958] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:14 2022] DELETE /identity/v3/projects/e3df00e25d7d49a98f98abc484148f92 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:15:14.925699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ada28-0760-4832-b06c-bd6a84d89fb1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:14.926001 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ada28-0760-4832-b06c-bd6a84d89fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:14.926269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ada28-0760-4832-b06c-bd6a84d89fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:14.940263 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-438db982-3193-4f5f-a462-c3374943ee9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e79bebf9bf524d1da6ad69326ffb78dc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:14.941328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-438db982-3193-4f5f-a462-c3374943ee9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e79bebf9bf524d1da6ad69326ffb78dc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:14.946999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5c5ada28-0760-4832-b06c-bd6a84d89fb1 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:14.948889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5c5ada28-0760-4832-b06c-bd6a84d89fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:14.955403 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 968/1959] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:14 2022] DELETE /identity/v3/users/e79bebf9bf524d1da6ad69326ffb78dc => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:15.014026 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 992/1960] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:15:15.022017 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee940b7-7fa8-4272-b3ab-449c93acccc4 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:15.022305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee940b7-7fa8-4272-b3ab-449c93acccc4 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:15.022579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee940b7-7fa8-4272-b3ab-449c93acccc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:15.043793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-9ee940b7-7fa8-4272-b3ab-449c93acccc4 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:15.045017 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9ee940b7-7fa8-4272-b3ab-449c93acccc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:15.107189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 969/1961] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:15.111717 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ada28-0760-4832-b06c-bd6a84d89fb1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:15.122040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ada28-0760-4832-b06c-bd6a84d89fb1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:15.126625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ea903-5abe-45ea-931b-844f5f069434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:15.127373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ea903-5abe-45ea-931b-844f5f069434 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:15.127643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ea903-5abe-45ea-931b-844f5f069434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:15.127908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ea903-5abe-45ea-931b-844f5f069434 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:15.129476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ea903-5abe-45ea-931b-844f5f069434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:15:15.129775 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ea903-5abe-45ea-931b-844f5f069434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:15.131416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ea903-5abe-45ea-931b-844f5f069434 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:15.144756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 993/1962] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:15:15.148866 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee940b7-7fa8-4272-b3ab-449c93acccc4 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:15.158618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee940b7-7fa8-4272-b3ab-449c93acccc4 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:15.163688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94ab19-5563-4a18-b759-06c1959287cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:15.168605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94ab19-5563-4a18-b759-06c1959287cb admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:15.168605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94ab19-5563-4a18-b759-06c1959287cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:15.168605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94ab19-5563-4a18-b759-06c1959287cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:15.168605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94ab19-5563-4a18-b759-06c1959287cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:15.168605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94ab19-5563-4a18-b759-06c1959287cb admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:15.255262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 970/1963] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:15 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:15.256855 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ea903-5abe-45ea-931b-844f5f069434 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:15.273126 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ea903-5abe-45ea-931b-844f5f069434 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:15.344465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12e269-d42d-423e-9a45-17578b114c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:15.345087 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12e269-d42d-423e-9a45-17578b114c23 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:15.345363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12e269-d42d-423e-9a45-17578b114c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:15.345632 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12e269-d42d-423e-9a45-17578b114c23 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:15.346263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12e269-d42d-423e-9a45-17578b114c23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:15.348243 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12e269-d42d-423e-9a45-17578b114c23 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:15.432993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94ab19-5563-4a18-b759-06c1959287cb admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:15.435075 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6a94ab19-5563-4a18-b759-06c1959287cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:15.451960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94ab19-5563-4a18-b759-06c1959287cb admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:15.460815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:15.466120 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42c494-511a-42cf-ab6b-40ed652d4a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:15.466485 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42c494-511a-42cf-ab6b-40ed652d4a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:15.466768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42c494-511a-42cf-ab6b-40ed652d4a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:15.470021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42c494-511a-42cf-ab6b-40ed652d4a68 admin admin] PATH_INFO: `/v3/projects/5cf299028032452bbe88e7cbf5c8879a` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:15.470021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42c494-511a-42cf-ab6b-40ed652d4a68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cf299028032452bbe88e7cbf5c8879a)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:15.470335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42c494-511a-42cf-ab6b-40ed652d4a68 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:15.473465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:15.485085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 994/1964] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:15 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:15.491168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12e269-d42d-423e-9a45-17578b114c23 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:15.505981 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12e269-d42d-423e-9a45-17578b114c23 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:15.510919 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786998d-b7a3-4cc7-8355-bd138ae8f377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:15.517125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786998d-b7a3-4cc7-8355-bd138ae8f377 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:15.517125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786998d-b7a3-4cc7-8355-bd138ae8f377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:15.517125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786998d-b7a3-4cc7-8355-bd138ae8f377 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:15.517125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786998d-b7a3-4cc7-8355-bd138ae8f377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:15.517125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786998d-b7a3-4cc7-8355-bd138ae8f377 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:15.527099 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 995/1965] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:15:15.531138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786998d-b7a3-4cc7-8355-bd138ae8f377 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:15.542713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786998d-b7a3-4cc7-8355-bd138ae8f377 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:15.548523 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780249a-9805-4c78-89ec-515ae5fb6562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:15.549143 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780249a-9805-4c78-89ec-515ae5fb6562 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:15.549414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780249a-9805-4c78-89ec-515ae5fb6562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:15.549680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780249a-9805-4c78-89ec-515ae5fb6562 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:15.550318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780249a-9805-4c78-89ec-515ae5fb6562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:15.551975 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780249a-9805-4c78-89ec-515ae5fb6562 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:15.566169 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 996/1966] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:15:15.567583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780249a-9805-4c78-89ec-515ae5fb6562 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:15.580162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780249a-9805-4c78-89ec-515ae5fb6562 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:15.592308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1232252-4395-4f42-82a7-1562826f4228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:15.592996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1232252-4395-4f42-82a7-1562826f4228 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:15.593384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1232252-4395-4f42-82a7-1562826f4228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:15.593715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1232252-4395-4f42-82a7-1562826f4228 admin admin] PATH_INFO: `/v3/projects/9b4d6dd628464c05b707fee1dd32f04a/users/e767c7081875428382af11d1ff475a20/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:15.614019 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1232252-4395-4f42-82a7-1562826f4228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b4d6dd628464c05b707fee1dd32f04a, user_id=e767c7081875428382af11d1ff475a20, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:15.617130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1232252-4395-4f42-82a7-1562826f4228 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:15.626382 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9a42c494-511a-42cf-ab6b-40ed652d4a68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cf299028032452bbe88e7cbf5c8879a', 'request_id': 'req-9a42c494-511a-42cf-ab6b-40ed652d4a68'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:15.633590 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9a42c494-511a-42cf-ab6b-40ed652d4a68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cf299028032452bbe88e7cbf5c8879a', 'request_id': 'req-9a42c494-511a-42cf-ab6b-40ed652d4a68'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:15.689220 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9a42c494-511a-42cf-ab6b-40ed652d4a68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cf299028032452bbe88e7cbf5c8879a', 'request_id': 'req-9a42c494-511a-42cf-ab6b-40ed652d4a68'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:15.695816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 997/1967] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:15 2022] PUT /identity/v3/projects/9b4d6dd628464c05b707fee1dd32f04a/users/e767c7081875428382af11d1ff475a20/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:15:15.700749 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cdd9a-29df-47b8-9fa8-89793fb2c1a7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:15.701051 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cdd9a-29df-47b8-9fa8-89793fb2c1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:15.701483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cdd9a-29df-47b8-9fa8-89793fb2c1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:15.702655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9a42c494-511a-42cf-ab6b-40ed652d4a68 admin admin] The token cache is being invalidate because project 5cf299028032452bbe88e7cbf5c8879a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:15.703056 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9a42c494-511a-42cf-ab6b-40ed652d4a68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:15.703881 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9a42c494-511a-42cf-ab6b-40ed652d4a68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:15.705757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 971/1968] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:15 2022] DELETE /identity/v3/projects/5cf299028032452bbe88e7cbf5c8879a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:15:15.711161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b415206-413a-4c84-b92d-484441e97fc1 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:15.711518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b415206-413a-4c84-b92d-484441e97fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:15.711879 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b415206-413a-4c84-b92d-484441e97fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:15.723181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-580cdd9a-29df-47b8-9fa8-89793fb2c1a7 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:15.724448 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-580cdd9a-29df-47b8-9fa8-89793fb2c1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:15.734649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-7b415206-413a-4c84-b92d-484441e97fc1 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:15.736970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7b415206-413a-4c84-b92d-484441e97fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:15.823791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 972/1969] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:15:15.824176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 998/1970] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:15:15.826057 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17cc09-4260-4780-832b-ae35a6cfb2fe None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:15.827334 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17cc09-4260-4780-832b-ae35a6cfb2fe None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:15.827548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580cdd9a-29df-47b8-9fa8-89793fb2c1a7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:15.827705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17cc09-4260-4780-832b-ae35a6cfb2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:15.829279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-580cdd9a-29df-47b8-9fa8-89793fb2c1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:15.851407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-1e17cc09-4260-4780-832b-ae35a6cfb2fe None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:15.852798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1e17cc09-4260-4780-832b-ae35a6cfb2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:15.867085 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 973/1971] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:15.871646 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17cc09-4260-4780-832b-ae35a6cfb2fe None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:15.887790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17cc09-4260-4780-832b-ae35a6cfb2fe None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:15.893742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf305b-8eb1-472a-b3d2-16ca8275d71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:15.894366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf305b-8eb1-472a-b3d2-16ca8275d71a admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:15.894648 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf305b-8eb1-472a-b3d2-16ca8275d71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:15.894923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf305b-8eb1-472a-b3d2-16ca8275d71a admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:15.895627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf305b-8eb1-472a-b3d2-16ca8275d71a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:15:15.895922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf305b-8eb1-472a-b3d2-16ca8275d71a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:15.898746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf305b-8eb1-472a-b3d2-16ca8275d71a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:15.906393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580cdd9a-29df-47b8-9fa8-89793fb2c1a7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:15.910231 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d95cd-7896-4123-98fe-a0bf6a3ac419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:15.911428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d95cd-7896-4123-98fe-a0bf6a3ac419 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:15.913837 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d95cd-7896-4123-98fe-a0bf6a3ac419 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:15.913837 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d95cd-7896-4123-98fe-a0bf6a3ac419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:15.918012 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 974/1972] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:15:15.922846 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d95cd-7896-4123-98fe-a0bf6a3ac419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:15.923961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf305b-8eb1-472a-b3d2-16ca8275d71a admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:15.924354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d95cd-7896-4123-98fe-a0bf6a3ac419 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:15.935474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 999/1973] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:15.937517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf305b-8eb1-472a-b3d2-16ca8275d71a admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:15.942568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970306d-b7b2-4ab8-a450-9d5dd582c90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:15.943200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970306d-b7b2-4ab8-a450-9d5dd582c90b admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:15.943474 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970306d-b7b2-4ab8-a450-9d5dd582c90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:15.943742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970306d-b7b2-4ab8-a450-9d5dd582c90b admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:15.944368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970306d-b7b2-4ab8-a450-9d5dd582c90b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:15.946478 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970306d-b7b2-4ab8-a450-9d5dd582c90b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:16.081530 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 975/1974] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:15 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:16.085346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d95cd-7896-4123-98fe-a0bf6a3ac419 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:16.096483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d95cd-7896-4123-98fe-a0bf6a3ac419 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:16.169980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:16.178372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24a848-0b7a-427c-a6cd-f6dfdba98836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:16.180798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24a848-0b7a-427c-a6cd-f6dfdba98836 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:16.180798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24a848-0b7a-427c-a6cd-f6dfdba98836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:16.180798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24a848-0b7a-427c-a6cd-f6dfdba98836 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:16.180798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24a848-0b7a-427c-a6cd-f6dfdba98836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:16.184592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24a848-0b7a-427c-a6cd-f6dfdba98836 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:16.302820 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1000/1975] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:16 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:15:16.307870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970306d-b7b2-4ab8-a450-9d5dd582c90b admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:16.322615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970306d-b7b2-4ab8-a450-9d5dd582c90b admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:16.332537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e381e-1e35-486f-ae8e-f13f3f1cc1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:16.333209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e381e-1e35-486f-ae8e-f13f3f1cc1fd admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:16.333532 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e381e-1e35-486f-ae8e-f13f3f1cc1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:16.333920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e381e-1e35-486f-ae8e-f13f3f1cc1fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:16.334923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e381e-1e35-486f-ae8e-f13f3f1cc1fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:16.336980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e381e-1e35-486f-ae8e-f13f3f1cc1fd admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:16.348817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 976/1976] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:15:16.353956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24a848-0b7a-427c-a6cd-f6dfdba98836 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:16.366105 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24a848-0b7a-427c-a6cd-f6dfdba98836 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:16.373987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d6dd9-f733-425c-93d8-1db7c7c0b6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:16.374975 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d6dd9-f733-425c-93d8-1db7c7c0b6f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:16.375430 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d6dd9-f733-425c-93d8-1db7c7c0b6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:16.375881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d6dd9-f733-425c-93d8-1db7c7c0b6f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:16.377135 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d6dd9-f733-425c-93d8-1db7c7c0b6f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:16.379924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d6dd9-f733-425c-93d8-1db7c7c0b6f2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:16.394900 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1001/1977] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:16.399401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e381e-1e35-486f-ae8e-f13f3f1cc1fd admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:16.415082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e381e-1e35-486f-ae8e-f13f3f1cc1fd admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:16.427443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cfe0d-be07-411a-9513-041e6a757e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:16.428563 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cfe0d-be07-411a-9513-041e6a757e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:16.429060 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cfe0d-be07-411a-9513-041e6a757e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:16.429553 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cfe0d-be07-411a-9513-041e6a757e58 admin admin] PATH_INFO: `/v3/projects/b9d304b7445b4b6583ee6b5cb70ff0a0/users/6a599cfa250d456dabb645dac26f692c/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:16.452372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7cfe0d-be07-411a-9513-041e6a757e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9d304b7445b4b6583ee6b5cb70ff0a0, user_id=6a599cfa250d456dabb645dac26f692c, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:16.455826 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7cfe0d-be07-411a-9513-041e6a757e58 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:16.546244 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 977/1978] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:16 2022] PUT /identity/v3/projects/b9d304b7445b4b6583ee6b5cb70ff0a0/users/6a599cfa250d456dabb645dac26f692c/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:15:16.547556 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d0dec-8b0a-4d1f-b156-9f385d063b01 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:16.548500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d0dec-8b0a-4d1f-b156-9f385d063b01 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:16.549004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d0dec-8b0a-4d1f-b156-9f385d063b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:16.583874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ae4d0dec-8b0a-4d1f-b156-9f385d063b01 None None] MFA Rules not processed for user `6a599cfa250d456dabb645dac26f692c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:16.591096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ae4d0dec-8b0a-4d1f-b156-9f385d063b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:16.683730 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1002/1979] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:15:16.688468 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cfe0d-be07-411a-9513-041e6a757e58 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:16.707848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:16.763122 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cfe0d-be07-411a-9513-041e6a757e58 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:16.769589 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa58c0-2abc-48af-b5e1-1230a0ca1e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:16.769898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa58c0-2abc-48af-b5e1-1230a0ca1e7f admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:16.770174 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa58c0-2abc-48af-b5e1-1230a0ca1e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:16.770445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa58c0-2abc-48af-b5e1-1230a0ca1e7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:16.771518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa58c0-2abc-48af-b5e1-1230a0ca1e7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:16.773553 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa58c0-2abc-48af-b5e1-1230a0ca1e7f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:16.882216 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 978/1980] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:16 2022] POST /identity/v3/projects => generated 381 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:16.884637 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d0dec-8b0a-4d1f-b156-9f385d063b01 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:16.896908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d0dec-8b0a-4d1f-b156-9f385d063b01 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:16.985137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867c421-ae5f-43de-8b8c-d20a3505f52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:16.985785 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867c421-ae5f-43de-8b8c-d20a3505f52e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:16.986055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867c421-ae5f-43de-8b8c-d20a3505f52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:16.986320 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867c421-ae5f-43de-8b8c-d20a3505f52e admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:16.986987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867c421-ae5f-43de-8b8c-d20a3505f52e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:16.989250 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867c421-ae5f-43de-8b8c-d20a3505f52e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:17.076450 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1003/1981] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:16 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:17.078469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa58c0-2abc-48af-b5e1-1230a0ca1e7f admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:17.090290 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa58c0-2abc-48af-b5e1-1230a0ca1e7f admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:17.106374 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48665d-4cc1-41d2-b531-81ed72fc4e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:17.106374 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48665d-4cc1-41d2-b531-81ed72fc4e04 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:17.106374 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48665d-4cc1-41d2-b531-81ed72fc4e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:17.106374 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48665d-4cc1-41d2-b531-81ed72fc4e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:17.107382 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48665d-4cc1-41d2-b531-81ed72fc4e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:17.107748 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48665d-4cc1-41d2-b531-81ed72fc4e04 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:17.125575 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 979/1982] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:17.127234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867c421-ae5f-43de-8b8c-d20a3505f52e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:17.140803 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867c421-ae5f-43de-8b8c-d20a3505f52e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:17.145526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1a580-2cff-460b-aa2f-0408fe4b3314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:17.146193 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1a580-2cff-460b-aa2f-0408fe4b3314 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:17.146758 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1a580-2cff-460b-aa2f-0408fe4b3314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:17.146758 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1a580-2cff-460b-aa2f-0408fe4b3314 admin admin] PATH_INFO: `/v3/projects/4a82ce9d9a3d4aa094c80aecac4d85a0/users/1dfd1563e87b4335a978ed97d89b34db/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:17.163359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1a580-2cff-460b-aa2f-0408fe4b3314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a82ce9d9a3d4aa094c80aecac4d85a0, user_id=1dfd1563e87b4335a978ed97d89b34db, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:17.165976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1a580-2cff-460b-aa2f-0408fe4b3314 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:17.268326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1004/1983] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:17 2022] PUT /identity/v3/projects/4a82ce9d9a3d4aa094c80aecac4d85a0/users/1dfd1563e87b4335a978ed97d89b34db/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:17.272257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48665d-4cc1-41d2-b531-81ed72fc4e04 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:17.283790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48665d-4cc1-41d2-b531-81ed72fc4e04 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:17.343028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f4be2-dcff-4a2d-93aa-98033a8020e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:17.344046 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f4be2-dcff-4a2d-93aa-98033a8020e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:17.349615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f4be2-dcff-4a2d-93aa-98033a8020e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:17.349615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f4be2-dcff-4a2d-93aa-98033a8020e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:17.349615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f4be2-dcff-4a2d-93aa-98033a8020e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:17.349615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f4be2-dcff-4a2d-93aa-98033a8020e8 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:17.363331 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 980/1984] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:17.364772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1a580-2cff-460b-aa2f-0408fe4b3314 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:17.379163 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1a580-2cff-460b-aa2f-0408fe4b3314 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:17.424477 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511fedf-c710-4395-89d8-4cb3303fe0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:17.424477 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511fedf-c710-4395-89d8-4cb3303fe0f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:17.424477 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511fedf-c710-4395-89d8-4cb3303fe0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:17.424477 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511fedf-c710-4395-89d8-4cb3303fe0f0 admin admin] PATH_INFO: `/v3/projects/4a82ce9d9a3d4aa094c80aecac4d85a0/users/1dfd1563e87b4335a978ed97d89b34db/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:17.424834 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511fedf-c710-4395-89d8-4cb3303fe0f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a82ce9d9a3d4aa094c80aecac4d85a0, user_id=1dfd1563e87b4335a978ed97d89b34db, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:17.424834 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511fedf-c710-4395-89d8-4cb3303fe0f0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:17.424834 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1005/1985] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:17 2022] PUT /identity/v3/projects/4a82ce9d9a3d4aa094c80aecac4d85a0/users/1dfd1563e87b4335a978ed97d89b34db/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:17.433762 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6610d2-744b-4d8f-b3fc-9972845c0720 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:17.434052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6610d2-744b-4d8f-b3fc-9972845c0720 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:17.434334 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6610d2-744b-4d8f-b3fc-9972845c0720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:17.467921 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-ae6610d2-744b-4d8f-b3fc-9972845c0720 None None] MFA Rules not processed for user `1dfd1563e87b4335a978ed97d89b34db`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:17.469075 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-ae6610d2-744b-4d8f-b3fc-9972845c0720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:17.578770 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 981/1986] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:17.587681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511fedf-c710-4395-89d8-4cb3303fe0f0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:17.599805 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c511fedf-c710-4395-89d8-4cb3303fe0f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:17.630931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:17.710517 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511fedf-c710-4395-89d8-4cb3303fe0f0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:17.715847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f98fbb-233a-4b91-a6b2-0eee283ddce8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:17.717274 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f98fbb-233a-4b91-a6b2-0eee283ddce8 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:17.718093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f98fbb-233a-4b91-a6b2-0eee283ddce8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:17.718093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f98fbb-233a-4b91-a6b2-0eee283ddce8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:17.730686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f98fbb-233a-4b91-a6b2-0eee283ddce8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:17.732289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f98fbb-233a-4b91-a6b2-0eee283ddce8 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:17.745958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1006/1987] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:18.040081 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6610d2-744b-4d8f-b3fc-9972845c0720 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:18.056193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6610d2-744b-4d8f-b3fc-9972845c0720 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:18.061294 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff94a79-4844-45c3-b9d0-50148dd914ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:18.062179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff94a79-4844-45c3-b9d0-50148dd914ba service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:18.062715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff94a79-4844-45c3-b9d0-50148dd914ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:18.063531 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff94a79-4844-45c3-b9d0-50148dd914ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:18.076885 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:18.159242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff94a79-4844-45c3-b9d0-50148dd914ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:18.160869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff94a79-4844-45c3-b9d0-50148dd914ba service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:18.178969 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 982/1988] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:18.759119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f98fbb-233a-4b91-a6b2-0eee283ddce8 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:18.877729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f98fbb-233a-4b91-a6b2-0eee283ddce8 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:18.894705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3b6e7-eac0-490b-a2a9-a4047569dea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:18.896571 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3b6e7-eac0-490b-a2a9-a4047569dea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:18.896571 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3b6e7-eac0-490b-a2a9-a4047569dea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:18.901965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3b6e7-eac0-490b-a2a9-a4047569dea8 admin admin] PATH_INFO: `/v3/users/6a599cfa250d456dabb645dac26f692c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:18.905654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3b6e7-eac0-490b-a2a9-a4047569dea8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a599cfa250d456dabb645dac26f692c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:18.912490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3b6e7-eac0-490b-a2a9-a4047569dea8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:19.150581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f1f3b6e7-eac0-490b-a2a9-a4047569dea8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a599cfa250d456dabb645dac26f692c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:19.180812 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f1f3b6e7-eac0-490b-a2a9-a4047569dea8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a599cfa250d456dabb645dac26f692c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:19.181624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f1f3b6e7-eac0-490b-a2a9-a4047569dea8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a599cfa250d456dabb645dac26f692c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:19.250600 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f1f3b6e7-eac0-490b-a2a9-a4047569dea8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a599cfa250d456dabb645dac26f692c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:19.251306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f1f3b6e7-eac0-490b-a2a9-a4047569dea8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a599cfa250d456dabb645dac26f692c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:19.279914 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1007/1989] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:18 2022] DELETE /identity/v3/users/6a599cfa250d456dabb645dac26f692c => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:19.281090 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087abf69-cc9f-4026-9498-8dfd839464ba None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:19.281885 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087abf69-cc9f-4026-9498-8dfd839464ba None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:19.289063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087abf69-cc9f-4026-9498-8dfd839464ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:19.319851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-087abf69-cc9f-4026-9498-8dfd839464ba None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:19.323799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-087abf69-cc9f-4026-9498-8dfd839464ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:19.392203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 983/1990] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:19.402832 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3b6e7-eac0-490b-a2a9-a4047569dea8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:19.404292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f1f3b6e7-eac0-490b-a2a9-a4047569dea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:19.469028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:19.508035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3b6e7-eac0-490b-a2a9-a4047569dea8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:19.512263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7fb2c-eabc-44ce-a99b-ee0be5cbb0ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:19.512973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7fb2c-eabc-44ce-a99b-ee0be5cbb0ac service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:19.513318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7fb2c-eabc-44ce-a99b-ee0be5cbb0ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:19.513677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7fb2c-eabc-44ce-a99b-ee0be5cbb0ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:19.526497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7fb2c-eabc-44ce-a99b-ee0be5cbb0ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:19.534343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7fb2c-eabc-44ce-a99b-ee0be5cbb0ac service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:19.552810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1008/1991] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:20.545344 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d675f-9b1e-4292-bc8a-aac5927c6457 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:20.545829 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d675f-9b1e-4292-bc8a-aac5927c6457 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:20.545910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d675f-9b1e-4292-bc8a-aac5927c6457 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:20.547514 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 984/1992] 10.222.0.30 () {54 vars in 921 bytes} [Thu Oct 6 08:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:15:20.622598 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa51043-25c7-4a4f-a9ef-5ccb95d7d3e3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:20.623140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa51043-25c7-4a4f-a9ef-5ccb95d7d3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:20.623219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa51043-25c7-4a4f-a9ef-5ccb95d7d3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:20.674386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3fa51043-25c7-4a4f-a9ef-5ccb95d7d3e3 None None] MFA Rules not processed for user `6074872303d3468fb566378fa34e8bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:20.677879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3fa51043-25c7-4a4f-a9ef-5ccb95d7d3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:20.756733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d675f-9b1e-4292-bc8a-aac5927c6457 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:20.759067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-eb2d675f-9b1e-4292-bc8a-aac5927c6457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:20.791175 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d675f-9b1e-4292-bc8a-aac5927c6457 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:20.814138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7926d-18df-4bfe-a462-34bad5dc0b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:20.815573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7926d-18df-4bfe-a462-34bad5dc0b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:20.819221 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7926d-18df-4bfe-a462-34bad5dc0b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:20.820113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7926d-18df-4bfe-a462-34bad5dc0b1b admin admin] PATH_INFO: `/v3/users/9427fe8723fb4b298b42106a862a798f` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:20.824966 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7926d-18df-4bfe-a462-34bad5dc0b1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9427fe8723fb4b298b42106a862a798f)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:20.828951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7926d-18df-4bfe-a462-34bad5dc0b1b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:20.895593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1009/1993] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Oct 6 08:15:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:20.911505 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc21794-a1a3-4f17-882b-b073fbd6e759 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:20.911938 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc21794-a1a3-4f17-882b-b073fbd6e759 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:20.912291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc21794-a1a3-4f17-882b-b073fbd6e759 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:20.914113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1010/1994] 10.222.0.30 () {54 vars in 961 bytes} [Thu Oct 6 08:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:15:20.922517 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5f2b8-f5b6-4f46-9e24-0a321c257a58 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:20.922806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5f2b8-f5b6-4f46-9e24-0a321c257a58 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:20.924183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5f2b8-f5b6-4f46-9e24-0a321c257a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:20.944164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bed5f2b8-f5b6-4f46-9e24-0a321c257a58 None None] MFA Rules not processed for user `6074872303d3468fb566378fa34e8bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:20.945437 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bed5f2b8-f5b6-4f46-9e24-0a321c257a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:20.957090 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1011/1995] 10.222.0.30 () {58 vars in 1058 bytes} [Thu Oct 6 08:15:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:20.964229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5f2b8-f5b6-4f46-9e24-0a321c257a58 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:20.977018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5f2b8-f5b6-4f46-9e24-0a321c257a58 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:20.988468 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869e061-6a2e-4b0b-8e73-367f0c76f1e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6074872303d3468fb566378fa34e8bea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:20.989086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869e061-6a2e-4b0b-8e73-367f0c76f1e3 service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:20.992549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869e061-6a2e-4b0b-8e73-367f0c76f1e3 service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:20.992549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869e061-6a2e-4b0b-8e73-367f0c76f1e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:21.003994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869e061-6a2e-4b0b-8e73-367f0c76f1e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:21.006676 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869e061-6a2e-4b0b-8e73-367f0c76f1e3 service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:21.022057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1012/1996] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:15:21.023129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869e061-6a2e-4b0b-8e73-367f0c76f1e3 service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:21.025607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3869e061-6a2e-4b0b-8e73-367f0c76f1e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:21.063121 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869e061-6a2e-4b0b-8e73-367f0c76f1e3 service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:21.068151 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce166977-3d4b-4ff8-a25e-d3c707575b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:21.069555 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce166977-3d4b-4ff8-a25e-d3c707575b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:21.069555 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce166977-3d4b-4ff8-a25e-d3c707575b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:21.069847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce166977-3d4b-4ff8-a25e-d3c707575b90 admin admin] PATH_INFO: `/v3/projects/b9d304b7445b4b6583ee6b5cb70ff0a0` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:21.070994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce166977-3d4b-4ff8-a25e-d3c707575b90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9d304b7445b4b6583ee6b5cb70ff0a0)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:21.074894 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce166977-3d4b-4ff8-a25e-d3c707575b90 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:21.097728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-edd7926d-18df-4bfe-a462-34bad5dc0b1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9427fe8723fb4b298b42106a862a798f'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:21.098453 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-edd7926d-18df-4bfe-a462-34bad5dc0b1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9427fe8723fb4b298b42106a862a798f'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:21.113782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-edd7926d-18df-4bfe-a462-34bad5dc0b1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9427fe8723fb4b298b42106a862a798f'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:21.236093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce166977-3d4b-4ff8-a25e-d3c707575b90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9d304b7445b4b6583ee6b5cb70ff0a0', 'request_id': 'req-ce166977-3d4b-4ff8-a25e-d3c707575b90'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:21.243091 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce166977-3d4b-4ff8-a25e-d3c707575b90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9d304b7445b4b6583ee6b5cb70ff0a0', 'request_id': 'req-ce166977-3d4b-4ff8-a25e-d3c707575b90'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:21.283061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-edd7926d-18df-4bfe-a462-34bad5dc0b1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9427fe8723fb4b298b42106a862a798f'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:21.284038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-edd7926d-18df-4bfe-a462-34bad5dc0b1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9427fe8723fb4b298b42106a862a798f'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:21.308340 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 985/1997] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:20 2022] DELETE /identity/v3/users/9427fe8723fb4b298b42106a862a798f => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:21.403843 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce166977-3d4b-4ff8-a25e-d3c707575b90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9d304b7445b4b6583ee6b5cb70ff0a0', 'request_id': 'req-ce166977-3d4b-4ff8-a25e-d3c707575b90'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:21.419364 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce166977-3d4b-4ff8-a25e-d3c707575b90 admin admin] The token cache is being invalidate because project b9d304b7445b4b6583ee6b5cb70ff0a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:21.419834 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce166977-3d4b-4ff8-a25e-d3c707575b90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:21.421244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ce166977-3d4b-4ff8-a25e-d3c707575b90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:21.426980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1013/1998] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:21 2022] DELETE /identity/v3/projects/b9d304b7445b4b6583ee6b5cb70ff0a0 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:21.581708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7926d-18df-4bfe-a462-34bad5dc0b1b admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:21.581708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-edd7926d-18df-4bfe-a462-34bad5dc0b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:21.581708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:21.581708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7926d-18df-4bfe-a462-34bad5dc0b1b admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:21.582714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95bf838-646c-4fb0-8b46-5abdf5cb7d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:21.582714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bf838-646c-4fb0-8b46-5abdf5cb7d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:21.582714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bf838-646c-4fb0-8b46-5abdf5cb7d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:21.582714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bf838-646c-4fb0-8b46-5abdf5cb7d21 admin admin] PATH_INFO: `/v3/users/1dfd1563e87b4335a978ed97d89b34db` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:21.583235 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95bf838-646c-4fb0-8b46-5abdf5cb7d21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dfd1563e87b4335a978ed97d89b34db)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:21.583235 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95bf838-646c-4fb0-8b46-5abdf5cb7d21 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:21.921666 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce166977-3d4b-4ff8-a25e-d3c707575b90 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:21.923894 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ce166977-3d4b-4ff8-a25e-d3c707575b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:22.011873 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:22.042500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce166977-3d4b-4ff8-a25e-d3c707575b90 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:22.046472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e95bf838-646c-4fb0-8b46-5abdf5cb7d21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dfd1563e87b4335a978ed97d89b34db'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:22.047505 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e95bf838-646c-4fb0-8b46-5abdf5cb7d21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dfd1563e87b4335a978ed97d89b34db'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:22.049303 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d32f4d-1e60-43c1-a217-6a9ad43f4c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:22.050186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d32f4d-1e60-43c1-a217-6a9ad43f4c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:22.051151 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d32f4d-1e60-43c1-a217-6a9ad43f4c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:22.051640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d32f4d-1e60-43c1-a217-6a9ad43f4c9b admin admin] PATH_INFO: `/v3/projects/37c1a79616054d3c988fac816f4acc8b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:22.053797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d32f4d-1e60-43c1-a217-6a9ad43f4c9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=37c1a79616054d3c988fac816f4acc8b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:22.058179 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d32f4d-1e60-43c1-a217-6a9ad43f4c9b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:22.073362 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e95bf838-646c-4fb0-8b46-5abdf5cb7d21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dfd1563e87b4335a978ed97d89b34db'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:22.262386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e95bf838-646c-4fb0-8b46-5abdf5cb7d21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dfd1563e87b4335a978ed97d89b34db'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:22.263181 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e95bf838-646c-4fb0-8b46-5abdf5cb7d21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dfd1563e87b4335a978ed97d89b34db'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:22.284690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 986/1999] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:21 2022] DELETE /identity/v3/users/1dfd1563e87b4335a978ed97d89b34db => generated 0 bytes in 854 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:22.285491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00d32f4d-1e60-43c1-a217-6a9ad43f4c9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37c1a79616054d3c988fac816f4acc8b', 'request_id': 'req-00d32f4d-1e60-43c1-a217-6a9ad43f4c9b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:22.292535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00d32f4d-1e60-43c1-a217-6a9ad43f4c9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37c1a79616054d3c988fac816f4acc8b', 'request_id': 'req-00d32f4d-1e60-43c1-a217-6a9ad43f4c9b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:22.392156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00d32f4d-1e60-43c1-a217-6a9ad43f4c9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37c1a79616054d3c988fac816f4acc8b', 'request_id': 'req-00d32f4d-1e60-43c1-a217-6a9ad43f4c9b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:22.420747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00d32f4d-1e60-43c1-a217-6a9ad43f4c9b admin admin] The token cache is being invalidate because project 37c1a79616054d3c988fac816f4acc8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:22.424909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00d32f4d-1e60-43c1-a217-6a9ad43f4c9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:22.424909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00d32f4d-1e60-43c1-a217-6a9ad43f4c9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:22.431494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1014/2000] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:21 2022] DELETE /identity/v3/projects/37c1a79616054d3c988fac816f4acc8b => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:22.433733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290cb34-a7c2-4cc2-9ed4-f1f2b8d54f83 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:22.434214 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290cb34-a7c2-4cc2-9ed4-f1f2b8d54f83 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:22.434788 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290cb34-a7c2-4cc2-9ed4-f1f2b8d54f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:22.476319 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-f290cb34-a7c2-4cc2-9ed4-f1f2b8d54f83 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:22.478105 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f290cb34-a7c2-4cc2-9ed4-f1f2b8d54f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:22.584910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 987/2001] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:22.593739 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e83f16-d7a6-477c-ae3b-0b920e809ef5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:22.594377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e83f16-d7a6-477c-ae3b-0b920e809ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:22.594757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e83f16-d7a6-477c-ae3b-0b920e809ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:22.627599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-43e83f16-d7a6-477c-ae3b-0b920e809ef5 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:22.627599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-43e83f16-d7a6-477c-ae3b-0b920e809ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:22.643344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1015/2002] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:22.648635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290cb34-a7c2-4cc2-9ed4-f1f2b8d54f83 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:22.687895 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290cb34-a7c2-4cc2-9ed4-f1f2b8d54f83 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:22.704175 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0862eb-7684-4104-a879-2d74084bf05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:22.705040 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0862eb-7684-4104-a879-2d74084bf05e admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:22.705613 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0862eb-7684-4104-a879-2d74084bf05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:22.705885 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0862eb-7684-4104-a879-2d74084bf05e admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:22.706552 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0862eb-7684-4104-a879-2d74084bf05e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:15:22.706857 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0862eb-7684-4104-a879-2d74084bf05e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:22.708547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0862eb-7684-4104-a879-2d74084bf05e admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:22.747101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e83f16-d7a6-477c-ae3b-0b920e809ef5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:22.755854 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 988/2003] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:15:22.759827 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e83f16-d7a6-477c-ae3b-0b920e809ef5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:22.766176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013a8e3-2041-4428-90bb-2aefdabafe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:22.767165 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013a8e3-2041-4428-90bb-2aefdabafe0b admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:22.767287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013a8e3-2041-4428-90bb-2aefdabafe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:22.767524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013a8e3-2041-4428-90bb-2aefdabafe0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:22.768170 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d013a8e3-2041-4428-90bb-2aefdabafe0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:22.770378 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d013a8e3-2041-4428-90bb-2aefdabafe0b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:22.912967 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1016/2004] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:22 2022] POST /identity/v3/projects => generated 367 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:22.913581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0862eb-7684-4104-a879-2d74084bf05e admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:22.924805 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0862eb-7684-4104-a879-2d74084bf05e admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:22.955257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:22.989797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce63d6f-9be8-47ed-ae53-0faae212f175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:22.990528 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce63d6f-9be8-47ed-ae53-0faae212f175 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:22.990876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce63d6f-9be8-47ed-ae53-0faae212f175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:22.991245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce63d6f-9be8-47ed-ae53-0faae212f175 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:22.991970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce63d6f-9be8-47ed-ae53-0faae212f175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:22.994983 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce63d6f-9be8-47ed-ae53-0faae212f175 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:23.120628 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 989/2005] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:22 2022] POST /identity/v3/users => generated 284 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:23.121734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013a8e3-2041-4428-90bb-2aefdabafe0b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:23.144984 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013a8e3-2041-4428-90bb-2aefdabafe0b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:23.153258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad695e93-39c9-498c-ad1d-8621fbedb1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:23.154283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad695e93-39c9-498c-ad1d-8621fbedb1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:23.154283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad695e93-39c9-498c-ad1d-8621fbedb1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:23.154479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad695e93-39c9-498c-ad1d-8621fbedb1d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:23.155474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad695e93-39c9-498c-ad1d-8621fbedb1d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:23.158149 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad695e93-39c9-498c-ad1d-8621fbedb1d4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:23.170865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1017/2006] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:23.174864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce63d6f-9be8-47ed-ae53-0faae212f175 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:23.187005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce63d6f-9be8-47ed-ae53-0faae212f175 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:23.192217 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54946f42-b566-4980-9494-dde749781414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:23.192957 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54946f42-b566-4980-9494-dde749781414 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:23.194707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54946f42-b566-4980-9494-dde749781414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:23.194707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54946f42-b566-4980-9494-dde749781414 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:23.194707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54946f42-b566-4980-9494-dde749781414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:23.196184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54946f42-b566-4980-9494-dde749781414 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:23.213015 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 990/2007] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:23.217829 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad695e93-39c9-498c-ad1d-8621fbedb1d4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:23.230498 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad695e93-39c9-498c-ad1d-8621fbedb1d4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:23.235454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8510a-6aa2-4ff9-a717-aaf9bd16e6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:23.236126 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8510a-6aa2-4ff9-a717-aaf9bd16e6d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:23.236401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8510a-6aa2-4ff9-a717-aaf9bd16e6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:23.236673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8510a-6aa2-4ff9-a717-aaf9bd16e6d3 admin admin] PATH_INFO: `/v3/projects/abf3523abba64f2ebe53a849e3aedf08/users/c121b4d484f04a08a0fb91a6eb94f1d6/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:23.252898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54946f42-b566-4980-9494-dde749781414 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:23.260013 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-54946f42-b566-4980-9494-dde749781414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:23.260443 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8510a-6aa2-4ff9-a717-aaf9bd16e6d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abf3523abba64f2ebe53a849e3aedf08, user_id=c121b4d484f04a08a0fb91a6eb94f1d6, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:23.262839 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8510a-6aa2-4ff9-a717-aaf9bd16e6d3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:23.293710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54946f42-b566-4980-9494-dde749781414 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:23.300606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba1130-c378-4739-b33a-5e65f116c548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:23.301344 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba1130-c378-4739-b33a-5e65f116c548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:23.301688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba1130-c378-4739-b33a-5e65f116c548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:23.302026 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba1130-c378-4739-b33a-5e65f116c548 admin admin] PATH_INFO: `/v3/projects/4a82ce9d9a3d4aa094c80aecac4d85a0` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:23.303315 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba1130-c378-4739-b33a-5e65f116c548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a82ce9d9a3d4aa094c80aecac4d85a0)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:23.305536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba1130-c378-4739-b33a-5e65f116c548 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:23.386576 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1018/2008] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:23 2022] PUT /identity/v3/projects/abf3523abba64f2ebe53a849e3aedf08/users/c121b4d484f04a08a0fb91a6eb94f1d6/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:15:23.389546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b5e9d-37c4-42fc-8469-7b9004ca5807 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:23.389827 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b5e9d-37c4-42fc-8469-7b9004ca5807 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:23.390095 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b5e9d-37c4-42fc-8469-7b9004ca5807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:23.459912 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a01b5e9d-37c4-42fc-8469-7b9004ca5807 None None] MFA Rules not processed for user `c121b4d484f04a08a0fb91a6eb94f1d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:23.461261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a01b5e9d-37c4-42fc-8469-7b9004ca5807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:23.483972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d4ba1130-c378-4739-b33a-5e65f116c548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a82ce9d9a3d4aa094c80aecac4d85a0', 'request_id': 'req-d4ba1130-c378-4739-b33a-5e65f116c548'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:23.513499 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d4ba1130-c378-4739-b33a-5e65f116c548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a82ce9d9a3d4aa094c80aecac4d85a0', 'request_id': 'req-d4ba1130-c378-4739-b33a-5e65f116c548'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:23.563610 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d4ba1130-c378-4739-b33a-5e65f116c548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a82ce9d9a3d4aa094c80aecac4d85a0', 'request_id': 'req-d4ba1130-c378-4739-b33a-5e65f116c548'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:23.582393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d4ba1130-c378-4739-b33a-5e65f116c548 admin admin] The token cache is being invalidate because project 4a82ce9d9a3d4aa094c80aecac4d85a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:23.582393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d4ba1130-c378-4739-b33a-5e65f116c548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:23.582393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d4ba1130-c378-4739-b33a-5e65f116c548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:23.584417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 991/2009] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:23 2022] DELETE /identity/v3/projects/4a82ce9d9a3d4aa094c80aecac4d85a0 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:23.592308 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe848b31-4bb7-4d7e-82b9-76bb93b39cdd None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:23.593023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe848b31-4bb7-4d7e-82b9-76bb93b39cdd None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:23.593023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe848b31-4bb7-4d7e-82b9-76bb93b39cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:23.600023 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:23.631948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1019/2010] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:23.642316 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b5e9d-37c4-42fc-8469-7b9004ca5807 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:23.642316 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a01b5e9d-37c4-42fc-8469-7b9004ca5807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:23.644044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-fe848b31-4bb7-4d7e-82b9-76bb93b39cdd None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:23.645266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-fe848b31-4bb7-4d7e-82b9-76bb93b39cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:23.737976 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 992/2011] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:23.739565 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee397a3-fa46-449a-ade8-81905d8cdf33 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:23.740043 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee397a3-fa46-449a-ade8-81905d8cdf33 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:23.740491 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee397a3-fa46-449a-ade8-81905d8cdf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:23.761196 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b5e9d-37c4-42fc-8469-7b9004ca5807 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:23.767497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce8602-b62a-4b28-b63b-75dcef13c06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:23.769010 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce8602-b62a-4b28-b63b-75dcef13c06b admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:23.774729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce8602-b62a-4b28-b63b-75dcef13c06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:23.774729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce8602-b62a-4b28-b63b-75dcef13c06b admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:23.774729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce8602-b62a-4b28-b63b-75dcef13c06b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:23.774729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce8602-b62a-4b28-b63b-75dcef13c06b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:23.785405 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-0ee397a3-fa46-449a-ade8-81905d8cdf33 None None] MFA Rules not processed for user `e767c7081875428382af11d1ff475a20`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:23.786681 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0ee397a3-fa46-449a-ade8-81905d8cdf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:23.810695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:23.855416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1020/2012] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:23 2022] POST /identity/v3/projects => generated 367 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:15:23.856875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c19334-0eab-471c-856e-789795fc751e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:23.857268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c19334-0eab-471c-856e-789795fc751e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:23.857663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c19334-0eab-471c-856e-789795fc751e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:23.885916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 993/2013] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:23.891066 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee397a3-fa46-449a-ade8-81905d8cdf33 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:23.891204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d8c19334-0eab-471c-856e-789795fc751e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:23.891204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d8c19334-0eab-471c-856e-789795fc751e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:23.899376 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee397a3-fa46-449a-ade8-81905d8cdf33 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:23.976595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1021/2014] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:23.976595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c19334-0eab-471c-856e-789795fc751e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:23.980681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d8c19334-0eab-471c-856e-789795fc751e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:23.981946 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829c68a-8c0a-4f13-b85d-be4f02044111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:23.982705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829c68a-8c0a-4f13-b85d-be4f02044111 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:23.983252 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829c68a-8c0a-4f13-b85d-be4f02044111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:23.983638 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829c68a-8c0a-4f13-b85d-be4f02044111 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:23.984375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829c68a-8c0a-4f13-b85d-be4f02044111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:23.986423 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829c68a-8c0a-4f13-b85d-be4f02044111 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:24.001493 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c19334-0eab-471c-856e-789795fc751e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:24.007836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86c4a6-1c02-4104-a9dd-8db530092361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:24.008531 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86c4a6-1c02-4104-a9dd-8db530092361 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:24.008804 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86c4a6-1c02-4104-a9dd-8db530092361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:24.009069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86c4a6-1c02-4104-a9dd-8db530092361 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:24.009783 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86c4a6-1c02-4104-a9dd-8db530092361 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:24.012574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86c4a6-1c02-4104-a9dd-8db530092361 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:24.090343 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 994/2015] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:23 2022] POST /identity/v3/users => generated 290 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:24.092145 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829c68a-8c0a-4f13-b85d-be4f02044111 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:24.106567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829c68a-8c0a-4f13-b85d-be4f02044111 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:24.117172 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafae7af-abd8-4fd9-b873-b6f4a4502846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:24.118183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafae7af-abd8-4fd9-b873-b6f4a4502846 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:24.118463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafae7af-abd8-4fd9-b873-b6f4a4502846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:24.120821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafae7af-abd8-4fd9-b873-b6f4a4502846 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:24.120821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafae7af-abd8-4fd9-b873-b6f4a4502846 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:15:24.120821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafae7af-abd8-4fd9-b873-b6f4a4502846 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:24.122602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafae7af-abd8-4fd9-b873-b6f4a4502846 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:24.132900 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1022/2016] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:23 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:24.136530 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86c4a6-1c02-4104-a9dd-8db530092361 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:24.140476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 995/2017] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:15:24.140476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafae7af-abd8-4fd9-b873-b6f4a4502846 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:24.159119 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafae7af-abd8-4fd9-b873-b6f4a4502846 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:24.218263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86c4a6-1c02-4104-a9dd-8db530092361 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:24.224179 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa21b2-35cd-4aa6-9309-494542a562a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:24.224963 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa21b2-35cd-4aa6-9309-494542a562a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:24.225520 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa21b2-35cd-4aa6-9309-494542a562a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:24.226004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa21b2-35cd-4aa6-9309-494542a562a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:24.227015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa21b2-35cd-4aa6-9309-494542a562a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:24.230813 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa21b2-35cd-4aa6-9309-494542a562a4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:24.232687 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:24.235797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895bc44d-f63e-487a-95b4-982d5476bca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:24.236486 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bc44d-f63e-487a-95b4-982d5476bca1 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:24.237204 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bc44d-f63e-487a-95b4-982d5476bca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:24.237601 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bc44d-f63e-487a-95b4-982d5476bca1 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:24.238905 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895bc44d-f63e-487a-95b4-982d5476bca1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:24.241331 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895bc44d-f63e-487a-95b4-982d5476bca1 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:24.248663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1023/2018] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:24.248663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa21b2-35cd-4aa6-9309-494542a562a4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:24.265885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa21b2-35cd-4aa6-9309-494542a562a4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:24.271332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bbd2e-c20b-4e3a-93a5-5a0fc7932d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:24.271961 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bbd2e-c20b-4e3a-93a5-5a0fc7932d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:24.272520 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bbd2e-c20b-4e3a-93a5-5a0fc7932d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:24.272520 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bbd2e-c20b-4e3a-93a5-5a0fc7932d9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:24.273142 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bbd2e-c20b-4e3a-93a5-5a0fc7932d9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:24.275207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297bbd2e-c20b-4e3a-93a5-5a0fc7932d9b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:24.380798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 996/2019] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:24 2022] POST /identity/v3/users => generated 298 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:24.381618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895bc44d-f63e-487a-95b4-982d5476bca1 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:24.392073 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895bc44d-f63e-487a-95b4-982d5476bca1 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:24.397004 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff1d78-e37e-47fc-8ebd-4aa448150741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:24.397704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff1d78-e37e-47fc-8ebd-4aa448150741 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:24.397979 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff1d78-e37e-47fc-8ebd-4aa448150741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:24.398249 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff1d78-e37e-47fc-8ebd-4aa448150741 admin admin] PATH_INFO: `/v3/projects/3fccf7def4fa4830b423da7719c57a9b/users/d5e0b024890244d0bccd01a1814962ce/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:24.413591 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1024/2020] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:24 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:24.413591 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bbd2e-c20b-4e3a-93a5-5a0fc7932d9b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:24.415215 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff1d78-e37e-47fc-8ebd-4aa448150741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fccf7def4fa4830b423da7719c57a9b, user_id=d5e0b024890244d0bccd01a1814962ce, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:24.419123 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff1d78-e37e-47fc-8ebd-4aa448150741 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:24.475315 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bbd2e-c20b-4e3a-93a5-5a0fc7932d9b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:24.480636 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24689e-db20-4b6e-a533-4da601c69a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:24.481236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24689e-db20-4b6e-a533-4da601c69a40 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:24.481513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24689e-db20-4b6e-a533-4da601c69a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:24.481783 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24689e-db20-4b6e-a533-4da601c69a40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:24.482573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24689e-db20-4b6e-a533-4da601c69a40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:24.484209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24689e-db20-4b6e-a533-4da601c69a40 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:24.494195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1025/2021] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:24.495026 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24689e-db20-4b6e-a533-4da601c69a40 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:24.509609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24689e-db20-4b6e-a533-4da601c69a40 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:24.515225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff9a7d-445a-4a37-81f5-6cda5266ab4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:24.515874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff9a7d-445a-4a37-81f5-6cda5266ab4a admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:24.516175 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff9a7d-445a-4a37-81f5-6cda5266ab4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:24.516448 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff9a7d-445a-4a37-81f5-6cda5266ab4a admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:24.517130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff9a7d-445a-4a37-81f5-6cda5266ab4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:24.519327 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff9a7d-445a-4a37-81f5-6cda5266ab4a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:24.587064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 997/2022] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:24 2022] PUT /identity/v3/projects/3fccf7def4fa4830b423da7719c57a9b/users/d5e0b024890244d0bccd01a1814962ce/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:24.589853 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff1d78-e37e-47fc-8ebd-4aa448150741 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:24.593825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:24.620341 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1026/2023] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:24 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:24.621271 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff9a7d-445a-4a37-81f5-6cda5266ab4a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:24.642539 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff9a7d-445a-4a37-81f5-6cda5266ab4a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:24.662825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff1d78-e37e-47fc-8ebd-4aa448150741 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:24.668319 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd4fca-5534-45aa-8145-9c2e7983aff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:24.668976 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd4fca-5534-45aa-8145-9c2e7983aff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:24.669245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd4fca-5534-45aa-8145-9c2e7983aff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:24.669511 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd4fca-5534-45aa-8145-9c2e7983aff7 admin admin] PATH_INFO: `/v3/projects/59fe95b503f5440bae2ed05569214c29/users/b0e5abb88fa74657a025bef294b74b91/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:24.684786 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd4fca-5534-45aa-8145-9c2e7983aff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59fe95b503f5440bae2ed05569214c29, user_id=b0e5abb88fa74657a025bef294b74b91, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:24.687583 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd4fca-5534-45aa-8145-9c2e7983aff7 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:24.714846 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffd438-9746-4787-a3d6-fc5a542eadbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:24.715516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffd438-9746-4787-a3d6-fc5a542eadbe admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:24.715785 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffd438-9746-4787-a3d6-fc5a542eadbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:24.716053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffd438-9746-4787-a3d6-fc5a542eadbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:24.716730 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffd438-9746-4787-a3d6-fc5a542eadbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:24.718314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffd438-9746-4787-a3d6-fc5a542eadbe admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:24.729540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1027/2024] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:24.730417 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffd438-9746-4787-a3d6-fc5a542eadbe admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:24.740521 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffd438-9746-4787-a3d6-fc5a542eadbe admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:24.745864 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c723d1e-3fe1-4f9c-b97b-fc908b9e483f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:24.746464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c723d1e-3fe1-4f9c-b97b-fc908b9e483f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:24.746738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c723d1e-3fe1-4f9c-b97b-fc908b9e483f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:24.747031 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c723d1e-3fe1-4f9c-b97b-fc908b9e483f admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:24.747671 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c723d1e-3fe1-4f9c-b97b-fc908b9e483f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:24.749248 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c723d1e-3fe1-4f9c-b97b-fc908b9e483f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:24.760518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1028/2025] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:24.761300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c723d1e-3fe1-4f9c-b97b-fc908b9e483f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:24.772580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c723d1e-3fe1-4f9c-b97b-fc908b9e483f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:24.777602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 998/2026] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:24 2022] PUT /identity/v3/projects/59fe95b503f5440bae2ed05569214c29/users/b0e5abb88fa74657a025bef294b74b91/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:24.780791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd4fca-5534-45aa-8145-9c2e7983aff7 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:24.814166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:24.842356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526dd65-262c-42be-9f68-e68613cea799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:24.843045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526dd65-262c-42be-9f68-e68613cea799 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:24.843396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526dd65-262c-42be-9f68-e68613cea799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:24.843577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526dd65-262c-42be-9f68-e68613cea799 admin admin] PATH_INFO: `/v3/projects/3fccf7def4fa4830b423da7719c57a9b/users/d5e0b024890244d0bccd01a1814962ce/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:24.845474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3526dd65-262c-42be-9f68-e68613cea799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fccf7def4fa4830b423da7719c57a9b, user_id=d5e0b024890244d0bccd01a1814962ce, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:24.848243 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3526dd65-262c-42be-9f68-e68613cea799 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:24.857593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd4fca-5534-45aa-8145-9c2e7983aff7 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:24.860479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1029/2027] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:24 2022] PUT /identity/v3/projects/3fccf7def4fa4830b423da7719c57a9b/users/d5e0b024890244d0bccd01a1814962ce/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:24.861448 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526dd65-262c-42be-9f68-e68613cea799 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:24.931758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870be21-1d9b-49f1-a9e8-66c55e41587d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:24.932793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870be21-1d9b-49f1-a9e8-66c55e41587d admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:24.933068 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870be21-1d9b-49f1-a9e8-66c55e41587d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:24.933341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870be21-1d9b-49f1-a9e8-66c55e41587d admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:24.933998 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870be21-1d9b-49f1-a9e8-66c55e41587d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:24.934762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526dd65-262c-42be-9f68-e68613cea799 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:24.935656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870be21-1d9b-49f1-a9e8-66c55e41587d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:24.939771 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c551a5-44ee-4c23-9998-1bb308e256d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:24.940622 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c551a5-44ee-4c23-9998-1bb308e256d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:24.940970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c551a5-44ee-4c23-9998-1bb308e256d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:24.941315 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c551a5-44ee-4c23-9998-1bb308e256d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:24.941971 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c551a5-44ee-4c23-9998-1bb308e256d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:24.943743 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c551a5-44ee-4c23-9998-1bb308e256d3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:24.949823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 999/2028] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:24.949823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928920e-42eb-4dda-8303-f30753aa8a16 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:24.949823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928920e-42eb-4dda-8303-f30753aa8a16 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:24.949823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928920e-42eb-4dda-8303-f30753aa8a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:24.955851 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1030/2029] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:24.956718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c551a5-44ee-4c23-9998-1bb308e256d3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:24.967954 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c551a5-44ee-4c23-9998-1bb308e256d3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:24.973258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70359b-82a6-4157-b443-31f30e284522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:24.973925 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70359b-82a6-4157-b443-31f30e284522 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:24.974204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70359b-82a6-4157-b443-31f30e284522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:24.974477 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70359b-82a6-4157-b443-31f30e284522 admin admin] PATH_INFO: `/v3/projects/e18ef65dd6c646e1b6154b7c44140efa/users/96e0bf53471c44079650554005aaf1ed/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:24.989285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-2928920e-42eb-4dda-8303-f30753aa8a16 None None] MFA Rules not processed for user `d5e0b024890244d0bccd01a1814962ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:24.990321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70359b-82a6-4157-b443-31f30e284522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18ef65dd6c646e1b6154b7c44140efa, user_id=96e0bf53471c44079650554005aaf1ed, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:24.990492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-2928920e-42eb-4dda-8303-f30753aa8a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:24.995334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70359b-82a6-4157-b443-31f30e284522 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:25.008637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:25.064004 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1000/2030] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:25.064714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928920e-42eb-4dda-8303-f30753aa8a16 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:25.082473 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928920e-42eb-4dda-8303-f30753aa8a16 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:25.087550 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1031/2031] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:24 2022] PUT /identity/v3/projects/e18ef65dd6c646e1b6154b7c44140efa/users/96e0bf53471c44079650554005aaf1ed/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:25.087550 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70359b-82a6-4157-b443-31f30e284522 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:25.087550 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2d70359b-82a6-4157-b443-31f30e284522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:25.181510 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d145857-41e7-46a8-a838-a322262b87eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:25.183292 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d145857-41e7-46a8-a838-a322262b87eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:25.183292 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d145857-41e7-46a8-a838-a322262b87eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:25.183292 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d145857-41e7-46a8-a838-a322262b87eb admin admin] PATH_INFO: `/v3/projects/59fe95b503f5440bae2ed05569214c29/users/b0e5abb88fa74657a025bef294b74b91/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:25.185196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d145857-41e7-46a8-a838-a322262b87eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=59fe95b503f5440bae2ed05569214c29, user_id=b0e5abb88fa74657a025bef294b74b91, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:25.187010 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70359b-82a6-4157-b443-31f30e284522 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:25.188423 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d145857-41e7-46a8-a838-a322262b87eb admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:25.191863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9862e87-2c2e-45c1-aafb-2ef08b6103d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:25.192277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9862e87-2c2e-45c1-aafb-2ef08b6103d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:25.192566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9862e87-2c2e-45c1-aafb-2ef08b6103d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:25.192866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9862e87-2c2e-45c1-aafb-2ef08b6103d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:25.201290 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1001/2032] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:25 2022] PUT /identity/v3/projects/59fe95b503f5440bae2ed05569214c29/users/b0e5abb88fa74657a025bef294b74b91/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:25.206106 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6209e-5f96-42b3-a48c-5d57246c3774 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:25.206444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6209e-5f96-42b3-a48c-5d57246c3774 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:25.206668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6209e-5f96-42b3-a48c-5d57246c3774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:25.246301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-46b6209e-5f96-42b3-a48c-5d57246c3774 None None] MFA Rules not processed for user `96e0bf53471c44079650554005aaf1ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:25.247960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-46b6209e-5f96-42b3-a48c-5d57246c3774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:25.274811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:25.278832 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9862e87-2c2e-45c1-aafb-2ef08b6103d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:25.280510 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9862e87-2c2e-45c1-aafb-2ef08b6103d4 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:25.296366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1032/2033] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:25.351045 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1002/2034] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:25.356540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9862e87-2c2e-45c1-aafb-2ef08b6103d4 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:25.367648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9862e87-2c2e-45c1-aafb-2ef08b6103d4 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:25.437569 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5885db8-bfad-4420-b9d0-997fe927e391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:25.439035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5885db8-bfad-4420-b9d0-997fe927e391 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:25.439035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5885db8-bfad-4420-b9d0-997fe927e391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:25.439615 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5885db8-bfad-4420-b9d0-997fe927e391 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:25.440586 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5885db8-bfad-4420-b9d0-997fe927e391 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:25.444519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5885db8-bfad-4420-b9d0-997fe927e391 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:25.591774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1033/2035] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:25 2022] POST /identity/v3/projects => generated 375 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:25.593521 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6209e-5f96-42b3-a48c-5d57246c3774 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:25.601566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5885db8-bfad-4420-b9d0-997fe927e391 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:25.615989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6209e-5f96-42b3-a48c-5d57246c3774 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:25.665303 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:25.685343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5885db8-bfad-4420-b9d0-997fe927e391 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:25.705072 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb08a4a-e85d-4fd0-a7fc-f993a0902f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:25.705821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb08a4a-e85d-4fd0-a7fc-f993a0902f91 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:25.706084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18e68f-9f61-4319-86f2-642029beaa30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:25.706276 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb08a4a-e85d-4fd0-a7fc-f993a0902f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:25.706656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb08a4a-e85d-4fd0-a7fc-f993a0902f91 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:25.706760 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18e68f-9f61-4319-86f2-642029beaa30 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:25.707061 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18e68f-9f61-4319-86f2-642029beaa30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:25.707332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18e68f-9f61-4319-86f2-642029beaa30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:25.707606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb08a4a-e85d-4fd0-a7fc-f993a0902f91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:25.710332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb08a4a-e85d-4fd0-a7fc-f993a0902f91 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:25.777569 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18e68f-9f61-4319-86f2-642029beaa30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:25.779525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18e68f-9f61-4319-86f2-642029beaa30 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:25.783950 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1003/2036] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:25 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:25.792685 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1034/2037] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:25.804472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb08a4a-e85d-4fd0-a7fc-f993a0902f91 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:25.839747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb08a4a-e85d-4fd0-a7fc-f993a0902f91 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:25.857929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427493b-e717-48b6-bd0e-1051ccaebc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:25.859073 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427493b-e717-48b6-bd0e-1051ccaebc45 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:25.859499 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427493b-e717-48b6-bd0e-1051ccaebc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:25.859866 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427493b-e717-48b6-bd0e-1051ccaebc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:25.860429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427493b-e717-48b6-bd0e-1051ccaebc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:25.863795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427493b-e717-48b6-bd0e-1051ccaebc45 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:25.900092 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18e68f-9f61-4319-86f2-642029beaa30 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:25.903188 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1004/2038] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:25.915455 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18e68f-9f61-4319-86f2-642029beaa30 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:25.927880 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897343fb-3d84-4d23-b89e-1735f5df1b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:25.927880 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897343fb-3d84-4d23-b89e-1735f5df1b49 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:25.927880 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897343fb-3d84-4d23-b89e-1735f5df1b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:25.928184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897343fb-3d84-4d23-b89e-1735f5df1b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:25.928631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897343fb-3d84-4d23-b89e-1735f5df1b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:25.933776 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897343fb-3d84-4d23-b89e-1735f5df1b49 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:25.954096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1035/2039] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:25.968315 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427493b-e717-48b6-bd0e-1051ccaebc45 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:25.980459 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427493b-e717-48b6-bd0e-1051ccaebc45 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:25.995401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfceb4-15bc-4a42-9fa4-c2d65cf31651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:25.996202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfceb4-15bc-4a42-9fa4-c2d65cf31651 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:25.996607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfceb4-15bc-4a42-9fa4-c2d65cf31651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:25.997418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfceb4-15bc-4a42-9fa4-c2d65cf31651 admin admin] PATH_INFO: `/v3/projects/c1539e0ea62f42009b021abcff87efaf/users/6ea4dff546564ce4a92d9482cfa32038/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:26.022326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfceb4-15bc-4a42-9fa4-c2d65cf31651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1539e0ea62f42009b021abcff87efaf, user_id=6ea4dff546564ce4a92d9482cfa32038, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:26.028109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfceb4-15bc-4a42-9fa4-c2d65cf31651 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:26.131960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1005/2040] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:25 2022] PUT /identity/v3/projects/c1539e0ea62f42009b021abcff87efaf/users/6ea4dff546564ce4a92d9482cfa32038/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:26.143529 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f93df-adef-4c61-82f4-72d5c0f2b02c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:26.143975 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f93df-adef-4c61-82f4-72d5c0f2b02c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:26.144108 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f93df-adef-4c61-82f4-72d5c0f2b02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:26.192518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0f4f93df-adef-4c61-82f4-72d5c0f2b02c None None] MFA Rules not processed for user `6ea4dff546564ce4a92d9482cfa32038`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:26.193994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0f4f93df-adef-4c61-82f4-72d5c0f2b02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:26.205019 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:26.270182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1036/2041] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:26.274488 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfceb4-15bc-4a42-9fa4-c2d65cf31651 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:26.347838 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfceb4-15bc-4a42-9fa4-c2d65cf31651 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:26.355198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e31c8a-130d-4cbe-9f6c-d2a3e3707fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:26.355874 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e31c8a-130d-4cbe-9f6c-d2a3e3707fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:26.356005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e31c8a-130d-4cbe-9f6c-d2a3e3707fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:26.356689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e31c8a-130d-4cbe-9f6c-d2a3e3707fc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:26.357375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e31c8a-130d-4cbe-9f6c-d2a3e3707fc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:26.359380 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e31c8a-130d-4cbe-9f6c-d2a3e3707fc1 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:26.480143 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1006/2042] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:26 2022] POST /identity/v3/projects => generated 377 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:26.481589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f93df-adef-4c61-82f4-72d5c0f2b02c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:26.492433 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f93df-adef-4c61-82f4-72d5c0f2b02c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:26.558298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cd804-a165-48dd-b768-7e2d516a25ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:26.559059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cd804-a165-48dd-b768-7e2d516a25ec admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:26.559404 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cd804-a165-48dd-b768-7e2d516a25ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:26.559743 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cd804-a165-48dd-b768-7e2d516a25ec admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:26.560513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5cd804-a165-48dd-b768-7e2d516a25ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:26.562561 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5cd804-a165-48dd-b768-7e2d516a25ec admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:26.673366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1037/2043] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:26 2022] POST /identity/v3/users => generated 295 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:26.677341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e31c8a-130d-4cbe-9f6c-d2a3e3707fc1 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:26.702966 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e31c8a-130d-4cbe-9f6c-d2a3e3707fc1 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:26.719148 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d3cf2-3f32-45ed-a3dc-61966f6cc5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:26.720053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d3cf2-3f32-45ed-a3dc-61966f6cc5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:26.720537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d3cf2-3f32-45ed-a3dc-61966f6cc5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:26.721027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d3cf2-3f32-45ed-a3dc-61966f6cc5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:26.722478 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d3cf2-3f32-45ed-a3dc-61966f6cc5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:26.724347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d3cf2-3f32-45ed-a3dc-61966f6cc5d8 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:26.739870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1007/2044] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:26.748244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cd804-a165-48dd-b768-7e2d516a25ec admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:26.769322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cd804-a165-48dd-b768-7e2d516a25ec admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:26.777799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee985d-7a00-4b3f-ae3e-5a375e635986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:26.781613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee985d-7a00-4b3f-ae3e-5a375e635986 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:26.781613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee985d-7a00-4b3f-ae3e-5a375e635986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:26.781613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee985d-7a00-4b3f-ae3e-5a375e635986 admin admin] PATH_INFO: `/v3/projects/1f05f015bcb84862b4c298c5a2fc5526/users/53b91190eedb459eaed23e55c1914a49/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:26.799857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee985d-7a00-4b3f-ae3e-5a375e635986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f05f015bcb84862b4c298c5a2fc5526, user_id=53b91190eedb459eaed23e55c1914a49, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:26.804468 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee985d-7a00-4b3f-ae3e-5a375e635986 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:26.902090 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1038/2045] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:26 2022] PUT /identity/v3/projects/1f05f015bcb84862b4c298c5a2fc5526/users/53b91190eedb459eaed23e55c1914a49/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:26.906775 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d3cf2-3f32-45ed-a3dc-61966f6cc5d8 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:26.918185 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d3cf2-3f32-45ed-a3dc-61966f6cc5d8 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:27.032024 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1abd63-7686-4ac0-9f56-4cf2504c20d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:27.032778 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1abd63-7686-4ac0-9f56-4cf2504c20d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:27.033188 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1abd63-7686-4ac0-9f56-4cf2504c20d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:27.033564 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1abd63-7686-4ac0-9f56-4cf2504c20d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:27.034303 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1abd63-7686-4ac0-9f56-4cf2504c20d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:27.036079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1abd63-7686-4ac0-9f56-4cf2504c20d7 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:27.047473 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1008/2046] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:27.053157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee985d-7a00-4b3f-ae3e-5a375e635986 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:27.075013 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee985d-7a00-4b3f-ae3e-5a375e635986 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:27.080782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966859ce-567d-4433-9ce5-af86602f5d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:27.081448 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966859ce-567d-4433-9ce5-af86602f5d7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:27.081721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966859ce-567d-4433-9ce5-af86602f5d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:27.081988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966859ce-567d-4433-9ce5-af86602f5d7f admin admin] PATH_INFO: `/v3/projects/1f05f015bcb84862b4c298c5a2fc5526/users/53b91190eedb459eaed23e55c1914a49/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:27.084674 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966859ce-567d-4433-9ce5-af86602f5d7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f05f015bcb84862b4c298c5a2fc5526, user_id=53b91190eedb459eaed23e55c1914a49, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:27.087671 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966859ce-567d-4433-9ce5-af86602f5d7f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:27.098859 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1039/2047] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:27 2022] PUT /identity/v3/projects/1f05f015bcb84862b4c298c5a2fc5526/users/53b91190eedb459eaed23e55c1914a49/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:15:27.110669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfc0f2-8fe0-4013-9f7d-17052524797a None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:27.111027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfc0f2-8fe0-4013-9f7d-17052524797a None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:27.111243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfc0f2-8fe0-4013-9f7d-17052524797a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:27.120988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:27.167851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-1adfc0f2-8fe0-4013-9f7d-17052524797a None None] MFA Rules not processed for user `53b91190eedb459eaed23e55c1914a49`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:27.169513 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1adfc0f2-8fe0-4013-9f7d-17052524797a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:27.278592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1009/2048] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:27.290789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966859ce-567d-4433-9ce5-af86602f5d7f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:27.293543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-966859ce-567d-4433-9ce5-af86602f5d7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:27.379430 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966859ce-567d-4433-9ce5-af86602f5d7f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:27.385111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d626d-d6b6-47c4-a375-9f6b73ec0644 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:27.386025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d626d-d6b6-47c4-a375-9f6b73ec0644 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:27.386304 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d626d-d6b6-47c4-a375-9f6b73ec0644 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:27.386573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d626d-d6b6-47c4-a375-9f6b73ec0644 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:27.408564 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:27.467353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d626d-d6b6-47c4-a375-9f6b73ec0644 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:27.468825 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d626d-d6b6-47c4-a375-9f6b73ec0644 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:27.497508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1040/2049] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:30.964872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfc0f2-8fe0-4013-9f7d-17052524797a None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:30.980823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfc0f2-8fe0-4013-9f7d-17052524797a None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:30.986200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e9fad-caa4-4c74-b6e6-8b489ed9eeff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:30.986818 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e9fad-caa4-4c74-b6e6-8b489ed9eeff service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:30.987130 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e9fad-caa4-4c74-b6e6-8b489ed9eeff service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:30.987400 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e9fad-caa4-4c74-b6e6-8b489ed9eeff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:31.057035 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e9fad-caa4-4c74-b6e6-8b489ed9eeff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:31.058684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e9fad-caa4-4c74-b6e6-8b489ed9eeff service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:31.071522 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1010/2050] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:31.729548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b90dc2-4c2c-46b8-8a50-35d20933e1f1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:31.730022 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b90dc2-4c2c-46b8-8a50-35d20933e1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:31.730367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b90dc2-4c2c-46b8-8a50-35d20933e1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:31.777421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-66b90dc2-4c2c-46b8-8a50-35d20933e1f1 None None] MFA Rules not processed for user `b0e5abb88fa74657a025bef294b74b91`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:31.778759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-66b90dc2-4c2c-46b8-8a50-35d20933e1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:31.874713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1041/2051] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:31.888818 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e9fad-caa4-4c74-b6e6-8b489ed9eeff service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:31.890563 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d10e9fad-caa4-4c74-b6e6-8b489ed9eeff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:31.986579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e9fad-caa4-4c74-b6e6-8b489ed9eeff service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:31.990618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25529f-1a41-40f0-ab93-b509f524bc68 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30909d152a5c4a1fac6c1a2765868177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:31.991261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25529f-1a41-40f0-ab93-b509f524bc68 service glance] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:31.991538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25529f-1a41-40f0-ab93-b509f524bc68 service glance] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:31.991809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25529f-1a41-40f0-ab93-b509f524bc68 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:32.016693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:32.063061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25529f-1a41-40f0-ab93-b509f524bc68 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:32.064580 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25529f-1a41-40f0-ab93-b509f524bc68 service glance] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:32.074768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1011/2052] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:32.478109 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a4cdb-4ca2-4c6f-afc5-93336d0610fa None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:32.480581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a4cdb-4ca2-4c6f-afc5-93336d0610fa None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:32.480581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a4cdb-4ca2-4c6f-afc5-93336d0610fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:32.499576 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-605a4cdb-4ca2-4c6f-afc5-93336d0610fa None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:32.501140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-605a4cdb-4ca2-4c6f-afc5-93336d0610fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:32.577913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1042/2053] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Oct 6 08:15:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:32.623841 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25529f-1a41-40f0-ab93-b509f524bc68 service glance] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:32.625279 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3e25529f-1a41-40f0-ab93-b509f524bc68 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:32.716741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25529f-1a41-40f0-ab93-b509f524bc68 service glance] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:32.722631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b14f0-e14c-42f4-b11f-4337e57ddab7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:32.723414 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b14f0-e14c-42f4-b11f-4337e57ddab7 service swift] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:32.723766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b14f0-e14c-42f4-b11f-4337e57ddab7 service swift] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:32.724147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b14f0-e14c-42f4-b11f-4337e57ddab7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:32.734858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b14f0-e14c-42f4-b11f-4337e57ddab7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:32.736712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b14f0-e14c-42f4-b11f-4337e57ddab7 service swift] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:32.749109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1012/2054] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:15:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:15:32.917559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a4cdb-4ca2-4c6f-afc5-93336d0610fa None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:32.931238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a4cdb-4ca2-4c6f-afc5-93336d0610fa None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:32.936633 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca54098-232a-429a-9a46-75b1b2eedbb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:32.937244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca54098-232a-429a-9a46-75b1b2eedbb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:32.937711 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca54098-232a-429a-9a46-75b1b2eedbb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:32.937980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca54098-232a-429a-9a46-75b1b2eedbb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:32.949964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca54098-232a-429a-9a46-75b1b2eedbb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:32.952000 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca54098-232a-429a-9a46-75b1b2eedbb9 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:32.968648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1043/2055] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:34.377376 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b14f0-e14c-42f4-b11f-4337e57ddab7 service swift] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:34.529721 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b14f0-e14c-42f4-b11f-4337e57ddab7 service swift] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:34.541412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa870d-916b-495d-adf7-6b5df465bce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:34.543695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa870d-916b-495d-adf7-6b5df465bce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:34.543695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa870d-916b-495d-adf7-6b5df465bce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:34.543695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa870d-916b-495d-adf7-6b5df465bce1 admin admin] PATH_INFO: `/v3/users/c121b4d484f04a08a0fb91a6eb94f1d6` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:34.544652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa870d-916b-495d-adf7-6b5df465bce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c121b4d484f04a08a0fb91a6eb94f1d6)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:34.547537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa870d-916b-495d-adf7-6b5df465bce1 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:34.765874 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-18fa870d-916b-495d-adf7-6b5df465bce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c121b4d484f04a08a0fb91a6eb94f1d6'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:34.767095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-18fa870d-916b-495d-adf7-6b5df465bce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c121b4d484f04a08a0fb91a6eb94f1d6'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:34.772569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:34.784752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-18fa870d-916b-495d-adf7-6b5df465bce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c121b4d484f04a08a0fb91a6eb94f1d6'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:34.877723 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-18fa870d-916b-495d-adf7-6b5df465bce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c121b4d484f04a08a0fb91a6eb94f1d6'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:34.878768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-18fa870d-916b-495d-adf7-6b5df465bce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c121b4d484f04a08a0fb91a6eb94f1d6'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:34.922154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1013/2056] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:34 2022] DELETE /identity/v3/users/c121b4d484f04a08a0fb91a6eb94f1d6 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:34.925374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064848b8-f90b-408e-8737-bc3daab87594 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:34.926413 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064848b8-f90b-408e-8737-bc3daab87594 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:34.927027 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064848b8-f90b-408e-8737-bc3daab87594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:34.957757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-064848b8-f90b-408e-8737-bc3daab87594 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:34.959352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-064848b8-f90b-408e-8737-bc3daab87594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:35.038339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:35.077995 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1044/2057] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:15:35.088833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa870d-916b-495d-adf7-6b5df465bce1 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:35.090586 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-18fa870d-916b-495d-adf7-6b5df465bce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:35.233775 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa870d-916b-495d-adf7-6b5df465bce1 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:35.238497 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3d922-6c11-4368-b135-ee46f841f6ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:35.239372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3d922-6c11-4368-b135-ee46f841f6ef service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:35.239828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3d922-6c11-4368-b135-ee46f841f6ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:35.240258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3d922-6c11-4368-b135-ee46f841f6ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:35.255753 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3d922-6c11-4368-b135-ee46f841f6ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:35.257806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3d922-6c11-4368-b135-ee46f841f6ef service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:35.274150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1014/2058] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:15:35.903734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064848b8-f90b-408e-8737-bc3daab87594 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:35.906725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-064848b8-f90b-408e-8737-bc3daab87594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:35.930179 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064848b8-f90b-408e-8737-bc3daab87594 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:35.948162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1026ba9-2eee-46b6-956a-2d93cfd877c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:35.949435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1026ba9-2eee-46b6-956a-2d93cfd877c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:35.951450 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1026ba9-2eee-46b6-956a-2d93cfd877c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:35.952621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1026ba9-2eee-46b6-956a-2d93cfd877c4 admin admin] PATH_INFO: `/v3/projects/abf3523abba64f2ebe53a849e3aedf08` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:35.957756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1026ba9-2eee-46b6-956a-2d93cfd877c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abf3523abba64f2ebe53a849e3aedf08)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:35.964842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1026ba9-2eee-46b6-956a-2d93cfd877c4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:36.169315 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c1026ba9-2eee-46b6-956a-2d93cfd877c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abf3523abba64f2ebe53a849e3aedf08', 'request_id': 'req-c1026ba9-2eee-46b6-956a-2d93cfd877c4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:36.184376 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c1026ba9-2eee-46b6-956a-2d93cfd877c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abf3523abba64f2ebe53a849e3aedf08', 'request_id': 'req-c1026ba9-2eee-46b6-956a-2d93cfd877c4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:36.255036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c1026ba9-2eee-46b6-956a-2d93cfd877c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abf3523abba64f2ebe53a849e3aedf08', 'request_id': 'req-c1026ba9-2eee-46b6-956a-2d93cfd877c4'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:36.288322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c1026ba9-2eee-46b6-956a-2d93cfd877c4 admin admin] The token cache is being invalidate because project abf3523abba64f2ebe53a849e3aedf08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:36.288873 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c1026ba9-2eee-46b6-956a-2d93cfd877c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:36.289930 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c1026ba9-2eee-46b6-956a-2d93cfd877c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:36.293934 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1045/2059] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:35 2022] DELETE /identity/v3/projects/abf3523abba64f2ebe53a849e3aedf08 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:36.308794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3d922-6c11-4368-b135-ee46f841f6ef service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:36.329844 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3d922-6c11-4368-b135-ee46f841f6ef service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:36.509377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e94b1e-8c3a-4a08-846b-b609d2c58238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:36.509377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e94b1e-8c3a-4a08-846b-b609d2c58238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:36.509377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e94b1e-8c3a-4a08-846b-b609d2c58238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:36.509377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e94b1e-8c3a-4a08-846b-b609d2c58238 admin admin] PATH_INFO: `/v3/users/d5e0b024890244d0bccd01a1814962ce` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:36.510176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e94b1e-8c3a-4a08-846b-b609d2c58238 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5e0b024890244d0bccd01a1814962ce)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:36.513684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e94b1e-8c3a-4a08-846b-b609d2c58238 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:36.969650 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-03e94b1e-8c3a-4a08-846b-b609d2c58238 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5e0b024890244d0bccd01a1814962ce'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:36.970326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-03e94b1e-8c3a-4a08-846b-b609d2c58238 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5e0b024890244d0bccd01a1814962ce'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:36.985768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-03e94b1e-8c3a-4a08-846b-b609d2c58238 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5e0b024890244d0bccd01a1814962ce'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:37.106163 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-03e94b1e-8c3a-4a08-846b-b609d2c58238 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5e0b024890244d0bccd01a1814962ce'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:37.109387 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-03e94b1e-8c3a-4a08-846b-b609d2c58238 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5e0b024890244d0bccd01a1814962ce'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:37.109947 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:37.124364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1015/2060] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:36 2022] DELETE /identity/v3/users/d5e0b024890244d0bccd01a1814962ce => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:37.731972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1026ba9-2eee-46b6-956a-2d93cfd877c4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:37.734085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c1026ba9-2eee-46b6-956a-2d93cfd877c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:37.764787 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1026ba9-2eee-46b6-956a-2d93cfd877c4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:37.874567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbfefb-0825-40e5-a865-f74c4658f9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:37.882613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbfefb-0825-40e5-a865-f74c4658f9e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:37.883369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbfefb-0825-40e5-a865-f74c4658f9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:37.883652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbfefb-0825-40e5-a865-f74c4658f9e1 admin admin] PATH_INFO: `/v3/projects/3fccf7def4fa4830b423da7719c57a9b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:37.886824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbfefb-0825-40e5-a865-f74c4658f9e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fccf7def4fa4830b423da7719c57a9b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:37.890139 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbfefb-0825-40e5-a865-f74c4658f9e1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:38.061461 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00fbfefb-0825-40e5-a865-f74c4658f9e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fccf7def4fa4830b423da7719c57a9b', 'request_id': 'req-00fbfefb-0825-40e5-a865-f74c4658f9e1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:38.076683 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00fbfefb-0825-40e5-a865-f74c4658f9e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fccf7def4fa4830b423da7719c57a9b', 'request_id': 'req-00fbfefb-0825-40e5-a865-f74c4658f9e1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:38.192772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00fbfefb-0825-40e5-a865-f74c4658f9e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fccf7def4fa4830b423da7719c57a9b', 'request_id': 'req-00fbfefb-0825-40e5-a865-f74c4658f9e1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:38.196822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:38.212212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00fbfefb-0825-40e5-a865-f74c4658f9e1 admin admin] The token cache is being invalidate because project 3fccf7def4fa4830b423da7719c57a9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:38.212504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00fbfefb-0825-40e5-a865-f74c4658f9e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:38.215306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00fbfefb-0825-40e5-a865-f74c4658f9e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:38.218192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1046/2061] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:37 2022] DELETE /identity/v3/projects/3fccf7def4fa4830b423da7719c57a9b => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:38.223496 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64400b-cdbf-4257-b266-35298bb7c2c3 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:38.224033 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64400b-cdbf-4257-b266-35298bb7c2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:38.224033 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64400b-cdbf-4257-b266-35298bb7c2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:38.254327 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-db64400b-cdbf-4257-b266-35298bb7c2c3 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:38.257038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-db64400b-cdbf-4257-b266-35298bb7c2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:38.370375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1016/2062] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:38.372149 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7631113-57c3-44d6-9a9c-2e888f537e7e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:38.372522 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7631113-57c3-44d6-9a9c-2e888f537e7e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:38.372874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7631113-57c3-44d6-9a9c-2e888f537e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:38.398369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b7631113-57c3-44d6-9a9c-2e888f537e7e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:38.401763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b7631113-57c3-44d6-9a9c-2e888f537e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:38.431023 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1047/2063] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:38.435893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64400b-cdbf-4257-b266-35298bb7c2c3 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:38.450107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64400b-cdbf-4257-b266-35298bb7c2c3 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:38.461647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b61040-3686-4f69-829d-529ac9f06548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:38.462283 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b61040-3686-4f69-829d-529ac9f06548 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:38.462562 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b61040-3686-4f69-829d-529ac9f06548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:38.462833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b61040-3686-4f69-829d-529ac9f06548 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:38.463745 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b61040-3686-4f69-829d-529ac9f06548 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:15:38.464069 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b61040-3686-4f69-829d-529ac9f06548 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:38.465689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b61040-3686-4f69-829d-529ac9f06548 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:38.484187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1017/2064] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:15:38.485206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7631113-57c3-44d6-9a9c-2e888f537e7e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:38.497800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7631113-57c3-44d6-9a9c-2e888f537e7e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:38.504507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc74ba4-042a-45e2-bfce-0f90a1efc61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:38.505095 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc74ba4-042a-45e2-bfce-0f90a1efc61f admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:38.505415 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc74ba4-042a-45e2-bfce-0f90a1efc61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:38.505642 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc74ba4-042a-45e2-bfce-0f90a1efc61f admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:38.506294 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc74ba4-042a-45e2-bfce-0f90a1efc61f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:38.508239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc74ba4-042a-45e2-bfce-0f90a1efc61f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:38.599468 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b61040-3686-4f69-829d-529ac9f06548 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:38.601439 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1048/2065] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:38 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:15:38.619373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b61040-3686-4f69-829d-529ac9f06548 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:38.725433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370f217-5a11-40d1-a931-282a628c1ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:38.726057 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370f217-5a11-40d1-a931-282a628c1ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:38.726891 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370f217-5a11-40d1-a931-282a628c1ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:38.727214 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370f217-5a11-40d1-a931-282a628c1ab3 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:38.727870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370f217-5a11-40d1-a931-282a628c1ab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:38.730703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370f217-5a11-40d1-a931-282a628c1ab3 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:38.922669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1018/2066] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:38 2022] POST /identity/v3/users => generated 291 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:15:38.926504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc74ba4-042a-45e2-bfce-0f90a1efc61f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:38.939917 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc74ba4-042a-45e2-bfce-0f90a1efc61f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:38.946595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b055c4-c9e3-4c09-8e15-81fe74f4c20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:38.946595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b055c4-c9e3-4c09-8e15-81fe74f4c20f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:38.946595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b055c4-c9e3-4c09-8e15-81fe74f4c20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:38.946595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b055c4-c9e3-4c09-8e15-81fe74f4c20f admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:38.947434 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b055c4-c9e3-4c09-8e15-81fe74f4c20f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:38.950404 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b055c4-c9e3-4c09-8e15-81fe74f4c20f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:38.965257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1049/2067] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:38.966843 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370f217-5a11-40d1-a931-282a628c1ab3 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:38.979165 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370f217-5a11-40d1-a931-282a628c1ab3 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:38.984103 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8072df5-638b-4957-8074-9c7475a4cae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:38.984711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8072df5-638b-4957-8074-9c7475a4cae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:38.985030 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8072df5-638b-4957-8074-9c7475a4cae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:38.985297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8072df5-638b-4957-8074-9c7475a4cae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:38.985937 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8072df5-638b-4957-8074-9c7475a4cae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:38.988098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8072df5-638b-4957-8074-9c7475a4cae7 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:39.002887 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1019/2068] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:39.006514 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b055c4-c9e3-4c09-8e15-81fe74f4c20f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:39.018067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b055c4-c9e3-4c09-8e15-81fe74f4c20f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:39.026127 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3c2f4-da5f-462c-8413-43eaf2046c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:39.026582 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3c2f4-da5f-462c-8413-43eaf2046c26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:39.026886 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3c2f4-da5f-462c-8413-43eaf2046c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:39.027186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3c2f4-da5f-462c-8413-43eaf2046c26 admin admin] PATH_INFO: `/v3/projects/2c9442fa1f5a448e88664f430f21fc43/users/81f03c4ac1644abebb634ee181d44b43/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:39.047814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3c2f4-da5f-462c-8413-43eaf2046c26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c9442fa1f5a448e88664f430f21fc43, user_id=81f03c4ac1644abebb634ee181d44b43, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:39.050553 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3c2f4-da5f-462c-8413-43eaf2046c26 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:39.149717 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1050/2069] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:39 2022] PUT /identity/v3/projects/2c9442fa1f5a448e88664f430f21fc43/users/81f03c4ac1644abebb634ee181d44b43/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:39.163138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a19f88-5af0-47dd-ae7c-884925f6c54f None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:39.163546 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a19f88-5af0-47dd-ae7c-884925f6c54f None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:39.163917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a19f88-5af0-47dd-ae7c-884925f6c54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:39.213975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-85a19f88-5af0-47dd-ae7c-884925f6c54f None None] MFA Rules not processed for user `81f03c4ac1644abebb634ee181d44b43`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:39.215364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-85a19f88-5af0-47dd-ae7c-884925f6c54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:39.286311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:39.328609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1020/2070] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:39.337689 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3c2f4-da5f-462c-8413-43eaf2046c26 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:39.452840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3c2f4-da5f-462c-8413-43eaf2046c26 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:39.458529 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e526f6f1-c2a0-4f98-b534-796b71d92803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:39.459611 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526f6f1-c2a0-4f98-b534-796b71d92803 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:39.460059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526f6f1-c2a0-4f98-b534-796b71d92803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:39.460465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526f6f1-c2a0-4f98-b534-796b71d92803 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:39.461340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e526f6f1-c2a0-4f98-b534-796b71d92803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:39.464311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e526f6f1-c2a0-4f98-b534-796b71d92803 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:39.582992 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1051/2071] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:39 2022] POST /identity/v3/projects => generated 381 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:39.586960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a19f88-5af0-47dd-ae7c-884925f6c54f None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:39.597707 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e526f6f1-c2a0-4f98-b534-796b71d92803 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:39.603316 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e526f6f1-c2a0-4f98-b534-796b71d92803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:39.606323 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a19f88-5af0-47dd-ae7c-884925f6c54f None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:39.714732 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d5572-1236-4234-9c90-0ddc90d6df17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:39.715783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d5572-1236-4234-9c90-0ddc90d6df17 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:39.716286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d5572-1236-4234-9c90-0ddc90d6df17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:39.716787 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d5572-1236-4234-9c90-0ddc90d6df17 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:39.717678 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d5572-1236-4234-9c90-0ddc90d6df17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:39.720671 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d5572-1236-4234-9c90-0ddc90d6df17 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:39.723508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e526f6f1-c2a0-4f98-b534-796b71d92803 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:39.729939 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971ee7a-26e6-4573-a3ee-3c8d90a44a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:39.730584 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971ee7a-26e6-4573-a3ee-3c8d90a44a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:39.730913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971ee7a-26e6-4573-a3ee-3c8d90a44a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:39.731283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971ee7a-26e6-4573-a3ee-3c8d90a44a3d admin admin] PATH_INFO: `/v3/users/96e0bf53471c44079650554005aaf1ed` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:39.732458 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971ee7a-26e6-4573-a3ee-3c8d90a44a3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=96e0bf53471c44079650554005aaf1ed)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:39.734590 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971ee7a-26e6-4573-a3ee-3c8d90a44a3d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:39.753067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:39.818650 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1021/2072] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:39 2022] POST /identity/v3/users => generated 297 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:39.823730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d5572-1236-4234-9c90-0ddc90d6df17 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:39.845158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d5572-1236-4234-9c90-0ddc90d6df17 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:39.855371 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6c5bc-61c8-4cc9-8f6c-2bf31ddde889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:39.856985 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6c5bc-61c8-4cc9-8f6c-2bf31ddde889 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:39.857460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6c5bc-61c8-4cc9-8f6c-2bf31ddde889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:39.857890 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6c5bc-61c8-4cc9-8f6c-2bf31ddde889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:39.859840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6c5bc-61c8-4cc9-8f6c-2bf31ddde889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:39.862721 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6c5bc-61c8-4cc9-8f6c-2bf31ddde889 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:39.885252 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1022/2073] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:39.889637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6c5bc-61c8-4cc9-8f6c-2bf31ddde889 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:39.903021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6c5bc-61c8-4cc9-8f6c-2bf31ddde889 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:39.909131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc037033-7339-4528-89b9-e1fdc8fc9132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:39.909931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc037033-7339-4528-89b9-e1fdc8fc9132 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:39.910976 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc037033-7339-4528-89b9-e1fdc8fc9132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:39.910976 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc037033-7339-4528-89b9-e1fdc8fc9132 admin admin] PATH_INFO: `/v3/projects/6e5a598e3aac46438eca22fcbd9f223c/users/8636ca57428f420d9572df6af5ae0233/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:39.942924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc037033-7339-4528-89b9-e1fdc8fc9132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e5a598e3aac46438eca22fcbd9f223c, user_id=8636ca57428f420d9572df6af5ae0233, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:39.949813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc037033-7339-4528-89b9-e1fdc8fc9132 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:39.993644 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c971ee7a-26e6-4573-a3ee-3c8d90a44a3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96e0bf53471c44079650554005aaf1ed'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:40.013729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c971ee7a-26e6-4573-a3ee-3c8d90a44a3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96e0bf53471c44079650554005aaf1ed'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:40.014277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c971ee7a-26e6-4573-a3ee-3c8d90a44a3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96e0bf53471c44079650554005aaf1ed'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:40.091757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1023/2074] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:39 2022] PUT /identity/v3/projects/6e5a598e3aac46438eca22fcbd9f223c/users/8636ca57428f420d9572df6af5ae0233/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:40.092649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc037033-7339-4528-89b9-e1fdc8fc9132 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:40.094471 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-bc037033-7339-4528-89b9-e1fdc8fc9132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:40.111963 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c971ee7a-26e6-4573-a3ee-3c8d90a44a3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96e0bf53471c44079650554005aaf1ed'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:40.112918 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c971ee7a-26e6-4573-a3ee-3c8d90a44a3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96e0bf53471c44079650554005aaf1ed'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:40.133675 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1052/2075] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:39 2022] DELETE /identity/v3/users/96e0bf53471c44079650554005aaf1ed => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:40.141821 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f10625-6143-4d86-bfc6-3daf71f16110 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:40.142468 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f10625-6143-4d86-bfc6-3daf71f16110 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:40.142839 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f10625-6143-4d86-bfc6-3daf71f16110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:40.168414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e9f10625-6143-4d86-bfc6-3daf71f16110 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:40.171843 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e9f10625-6143-4d86-bfc6-3daf71f16110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:40.219719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc037033-7339-4528-89b9-e1fdc8fc9132 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:40.288225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1053/2076] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:40.289632 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:40.298097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f10625-6143-4d86-bfc6-3daf71f16110 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:40.300085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e9f10625-6143-4d86-bfc6-3daf71f16110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:40.302732 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3c1dd-312a-40e3-bb22-c004fc32c009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:40.303398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3c1dd-312a-40e3-bb22-c004fc32c009 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:40.303683 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3c1dd-312a-40e3-bb22-c004fc32c009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:40.303960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3c1dd-312a-40e3-bb22-c004fc32c009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:40.304643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3c1dd-312a-40e3-bb22-c004fc32c009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:40.306255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3c1dd-312a-40e3-bb22-c004fc32c009 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:40.322998 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1024/2077] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:40.326654 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3c1dd-312a-40e3-bb22-c004fc32c009 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:40.340807 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3c1dd-312a-40e3-bb22-c004fc32c009 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:40.346016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe16b43-162d-4c6a-ae26-6469385c617d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:40.346672 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe16b43-162d-4c6a-ae26-6469385c617d admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:40.347266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe16b43-162d-4c6a-ae26-6469385c617d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:40.347266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe16b43-162d-4c6a-ae26-6469385c617d admin admin] PATH_INFO: `/v3/projects/6e5a598e3aac46438eca22fcbd9f223c/users/8636ca57428f420d9572df6af5ae0233/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:40.349262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe16b43-162d-4c6a-ae26-6469385c617d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e5a598e3aac46438eca22fcbd9f223c, user_id=8636ca57428f420d9572df6af5ae0233, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:40.352114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe16b43-162d-4c6a-ae26-6469385c617d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:40.371194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1025/2078] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:40 2022] PUT /identity/v3/projects/6e5a598e3aac46438eca22fcbd9f223c/users/8636ca57428f420d9572df6af5ae0233/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:40.384392 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6999b51-0b1b-41ff-b8ea-c6886be040a5 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:40.384766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6999b51-0b1b-41ff-b8ea-c6886be040a5 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:40.385722 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6999b51-0b1b-41ff-b8ea-c6886be040a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:40.406550 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f10625-6143-4d86-bfc6-3daf71f16110 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:40.460268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-f6999b51-0b1b-41ff-b8ea-c6886be040a5 None None] MFA Rules not processed for user `8636ca57428f420d9572df6af5ae0233`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:40.463126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f6999b51-0b1b-41ff-b8ea-c6886be040a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:40.491441 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:40.522800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ff956-8edc-42e8-a8fe-47def5dfc494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:40.523466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ff956-8edc-42e8-a8fe-47def5dfc494 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:40.523742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ff956-8edc-42e8-a8fe-47def5dfc494 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:40.524085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ff956-8edc-42e8-a8fe-47def5dfc494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:40.639719 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ff956-8edc-42e8-a8fe-47def5dfc494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:40.645303 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1026/2079] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:40.653120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ff956-8edc-42e8-a8fe-47def5dfc494 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:40.658789 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6999b51-0b1b-41ff-b8ea-c6886be040a5 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:40.658789 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f6999b51-0b1b-41ff-b8ea-c6886be040a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:40.674445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1054/2080] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:40.764561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6999b51-0b1b-41ff-b8ea-c6886be040a5 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:40.769153 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42020686-abca-4f9c-afb7-93fd6c8ede61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:40.769784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42020686-abca-4f9c-afb7-93fd6c8ede61 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:40.794347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42020686-abca-4f9c-afb7-93fd6c8ede61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:40.794347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42020686-abca-4f9c-afb7-93fd6c8ede61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:40.794347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42020686-abca-4f9c-afb7-93fd6c8ede61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:40.794347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42020686-abca-4f9c-afb7-93fd6c8ede61 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:40.794347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1027/2081] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:15:41.691628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ff956-8edc-42e8-a8fe-47def5dfc494 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:41.696707 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-008ff956-8edc-42e8-a8fe-47def5dfc494 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:41.719028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ff956-8edc-42e8-a8fe-47def5dfc494 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:41.726169 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ae493-a11a-4274-a81f-9550d349a489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:41.726852 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ae493-a11a-4274-a81f-9550d349a489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:41.727215 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ae493-a11a-4274-a81f-9550d349a489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:41.727549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ae493-a11a-4274-a81f-9550d349a489 admin admin] PATH_INFO: `/v3/projects/e18ef65dd6c646e1b6154b7c44140efa` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:41.728884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ae493-a11a-4274-a81f-9550d349a489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e18ef65dd6c646e1b6154b7c44140efa)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:41.731250 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ae493-a11a-4274-a81f-9550d349a489 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:41.979765 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42020686-abca-4f9c-afb7-93fd6c8ede61 service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:41.990581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b44ae493-a11a-4274-a81f-9550d349a489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e18ef65dd6c646e1b6154b7c44140efa', 'request_id': 'req-b44ae493-a11a-4274-a81f-9550d349a489'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:41.991520 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42020686-abca-4f9c-afb7-93fd6c8ede61 service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:41.998391 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b44ae493-a11a-4274-a81f-9550d349a489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e18ef65dd6c646e1b6154b7c44140efa', 'request_id': 'req-b44ae493-a11a-4274-a81f-9550d349a489'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:42.055352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b44ae493-a11a-4274-a81f-9550d349a489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e18ef65dd6c646e1b6154b7c44140efa', 'request_id': 'req-b44ae493-a11a-4274-a81f-9550d349a489'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:42.065837 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e19b04-8746-4ec7-a980-0e34c173ce52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:42.066569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e19b04-8746-4ec7-a980-0e34c173ce52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:42.067021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e19b04-8746-4ec7-a980-0e34c173ce52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:42.067607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e19b04-8746-4ec7-a980-0e34c173ce52 admin admin] PATH_INFO: `/v3/users/81f03c4ac1644abebb634ee181d44b43` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:42.068928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e19b04-8746-4ec7-a980-0e34c173ce52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81f03c4ac1644abebb634ee181d44b43)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:42.071080 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e19b04-8746-4ec7-a980-0e34c173ce52 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:42.073143 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:42.079115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b44ae493-a11a-4274-a81f-9550d349a489 admin admin] The token cache is being invalidate because project e18ef65dd6c646e1b6154b7c44140efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:42.079441 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b44ae493-a11a-4274-a81f-9550d349a489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:42.080161 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b44ae493-a11a-4274-a81f-9550d349a489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:42.082019 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1055/2082] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:41 2022] DELETE /identity/v3/projects/e18ef65dd6c646e1b6154b7c44140efa => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:42.086048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ae493-a11a-4274-a81f-9550d349a489 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:42.088325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b44ae493-a11a-4274-a81f-9550d349a489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:42.107342 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ae493-a11a-4274-a81f-9550d349a489 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:42.112972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212874c-a311-4a8a-b9e3-05daa2577f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:42.113742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212874c-a311-4a8a-b9e3-05daa2577f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:42.114107 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212874c-a311-4a8a-b9e3-05daa2577f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:42.114379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212874c-a311-4a8a-b9e3-05daa2577f51 admin admin] PATH_INFO: `/v3/users/6ea4dff546564ce4a92d9482cfa32038` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:42.115697 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212874c-a311-4a8a-b9e3-05daa2577f51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ea4dff546564ce4a92d9482cfa32038)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:42.117631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212874c-a311-4a8a-b9e3-05daa2577f51 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:42.295239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-49e19b04-8746-4ec7-a980-0e34c173ce52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81f03c4ac1644abebb634ee181d44b43'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:42.295934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-49e19b04-8746-4ec7-a980-0e34c173ce52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81f03c4ac1644abebb634ee181d44b43'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:42.310430 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-49e19b04-8746-4ec7-a980-0e34c173ce52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81f03c4ac1644abebb634ee181d44b43'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:42.369673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2212874c-a311-4a8a-b9e3-05daa2577f51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ea4dff546564ce4a92d9482cfa32038'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:42.382888 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2212874c-a311-4a8a-b9e3-05daa2577f51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ea4dff546564ce4a92d9482cfa32038'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:42.383636 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2212874c-a311-4a8a-b9e3-05daa2577f51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ea4dff546564ce4a92d9482cfa32038'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:42.426573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-49e19b04-8746-4ec7-a980-0e34c173ce52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81f03c4ac1644abebb634ee181d44b43'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:42.427467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-49e19b04-8746-4ec7-a980-0e34c173ce52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81f03c4ac1644abebb634ee181d44b43'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:42.450099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1028/2083] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:41 2022] DELETE /identity/v3/users/81f03c4ac1644abebb634ee181d44b43 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:15:42.451712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01747b-1a6f-4a4b-aa0a-cd1800f873a3 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:42.452190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01747b-1a6f-4a4b-aa0a-cd1800f873a3 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:42.452631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01747b-1a6f-4a4b-aa0a-cd1800f873a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:42.478524 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-2f01747b-1a6f-4a4b-aa0a-cd1800f873a3 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:42.479778 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-2f01747b-1a6f-4a4b-aa0a-cd1800f873a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:42.488570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2212874c-a311-4a8a-b9e3-05daa2577f51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ea4dff546564ce4a92d9482cfa32038'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:42.489390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2212874c-a311-4a8a-b9e3-05daa2577f51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ea4dff546564ce4a92d9482cfa32038'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:42.492467 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:42.522425 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1056/2084] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:42 2022] DELETE /identity/v3/users/6ea4dff546564ce4a92d9482cfa32038 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:42.588863 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1029/2085] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:42.607590 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212874c-a311-4a8a-b9e3-05daa2577f51 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:42.609536 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2212874c-a311-4a8a-b9e3-05daa2577f51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:42.709607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212874c-a311-4a8a-b9e3-05daa2577f51 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:42.714690 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590881ed-5098-4858-bd27-8b2d7e7a31d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:42.715466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590881ed-5098-4858-bd27-8b2d7e7a31d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:42.715741 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590881ed-5098-4858-bd27-8b2d7e7a31d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:42.716009 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590881ed-5098-4858-bd27-8b2d7e7a31d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:42.817842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590881ed-5098-4858-bd27-8b2d7e7a31d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:42.820233 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590881ed-5098-4858-bd27-8b2d7e7a31d7 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:42.836933 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1057/2086] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:43.104946 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01747b-1a6f-4a4b-aa0a-cd1800f873a3 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:43.106719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-2f01747b-1a6f-4a4b-aa0a-cd1800f873a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:43.123165 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01747b-1a6f-4a4b-aa0a-cd1800f873a3 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:43.129870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03dc65-12e8-4911-970f-9403e209a184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:43.132051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03dc65-12e8-4911-970f-9403e209a184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:43.132491 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03dc65-12e8-4911-970f-9403e209a184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:43.132916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03dc65-12e8-4911-970f-9403e209a184 admin admin] PATH_INFO: `/v3/projects/c1539e0ea62f42009b021abcff87efaf` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:43.136153 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03dc65-12e8-4911-970f-9403e209a184 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1539e0ea62f42009b021abcff87efaf)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:43.138529 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03dc65-12e8-4911-970f-9403e209a184 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:43.293300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fe03dc65-12e8-4911-970f-9403e209a184 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1539e0ea62f42009b021abcff87efaf', 'request_id': 'req-fe03dc65-12e8-4911-970f-9403e209a184'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:43.300032 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fe03dc65-12e8-4911-970f-9403e209a184 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1539e0ea62f42009b021abcff87efaf', 'request_id': 'req-fe03dc65-12e8-4911-970f-9403e209a184'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:43.380966 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fe03dc65-12e8-4911-970f-9403e209a184 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1539e0ea62f42009b021abcff87efaf', 'request_id': 'req-fe03dc65-12e8-4911-970f-9403e209a184'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:43.396892 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fe03dc65-12e8-4911-970f-9403e209a184 admin admin] The token cache is being invalidate because project c1539e0ea62f42009b021abcff87efaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:43.397298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fe03dc65-12e8-4911-970f-9403e209a184 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:43.399946 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fe03dc65-12e8-4911-970f-9403e209a184 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:43.410041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1030/2087] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:43 2022] DELETE /identity/v3/projects/c1539e0ea62f42009b021abcff87efaf => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:43.421948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590881ed-5098-4858-bd27-8b2d7e7a31d7 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:43.433305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590881ed-5098-4858-bd27-8b2d7e7a31d7 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:43.506745 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc825f79-2aa8-4197-b0ff-7e96a3af9878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:43.507780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc825f79-2aa8-4197-b0ff-7e96a3af9878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:43.508057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc825f79-2aa8-4197-b0ff-7e96a3af9878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:43.508356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc825f79-2aa8-4197-b0ff-7e96a3af9878 admin admin] PATH_INFO: `/v3/users/53b91190eedb459eaed23e55c1914a49` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:43.509626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc825f79-2aa8-4197-b0ff-7e96a3af9878 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53b91190eedb459eaed23e55c1914a49)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:43.512241 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc825f79-2aa8-4197-b0ff-7e96a3af9878 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:43.528388 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03dc65-12e8-4911-970f-9403e209a184 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:43.529994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-fe03dc65-12e8-4911-970f-9403e209a184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:43.544367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:43.553897 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03dc65-12e8-4911-970f-9403e209a184 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:43.571489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fda3de-ca8f-4f01-824c-68fe0c03c177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:43.572159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fda3de-ca8f-4f01-824c-68fe0c03c177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:43.581102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fda3de-ca8f-4f01-824c-68fe0c03c177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:43.581102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fda3de-ca8f-4f01-824c-68fe0c03c177 admin admin] PATH_INFO: `/v3/projects/2c9442fa1f5a448e88664f430f21fc43` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:43.581102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fda3de-ca8f-4f01-824c-68fe0c03c177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c9442fa1f5a448e88664f430f21fc43)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:43.581102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fda3de-ca8f-4f01-824c-68fe0c03c177 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:43.765506 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:43.776453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dc825f79-2aa8-4197-b0ff-7e96a3af9878 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53b91190eedb459eaed23e55c1914a49'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:43.779150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a7fda3de-ca8f-4f01-824c-68fe0c03c177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c9442fa1f5a448e88664f430f21fc43', 'request_id': 'req-a7fda3de-ca8f-4f01-824c-68fe0c03c177'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:43.787863 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a7fda3de-ca8f-4f01-824c-68fe0c03c177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c9442fa1f5a448e88664f430f21fc43', 'request_id': 'req-a7fda3de-ca8f-4f01-824c-68fe0c03c177'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:43.803196 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dc825f79-2aa8-4197-b0ff-7e96a3af9878 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53b91190eedb459eaed23e55c1914a49'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:43.804617 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dc825f79-2aa8-4197-b0ff-7e96a3af9878 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53b91190eedb459eaed23e55c1914a49'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:43.845378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a7fda3de-ca8f-4f01-824c-68fe0c03c177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c9442fa1f5a448e88664f430f21fc43', 'request_id': 'req-a7fda3de-ca8f-4f01-824c-68fe0c03c177'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:43.868929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a7fda3de-ca8f-4f01-824c-68fe0c03c177 admin admin] The token cache is being invalidate because project 2c9442fa1f5a448e88664f430f21fc43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:43.869914 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a7fda3de-ca8f-4f01-824c-68fe0c03c177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:43.870658 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a7fda3de-ca8f-4f01-824c-68fe0c03c177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:43.872240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1031/2088] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:43 2022] DELETE /identity/v3/projects/2c9442fa1f5a448e88664f430f21fc43 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:15:43.876137 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fda3de-ca8f-4f01-824c-68fe0c03c177 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:43.877448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a7fda3de-ca8f-4f01-824c-68fe0c03c177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:43.911415 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dc825f79-2aa8-4197-b0ff-7e96a3af9878 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53b91190eedb459eaed23e55c1914a49'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:43.912334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dc825f79-2aa8-4197-b0ff-7e96a3af9878 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53b91190eedb459eaed23e55c1914a49'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:43.952754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1058/2089] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:43 2022] DELETE /identity/v3/users/53b91190eedb459eaed23e55c1914a49 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:43.987329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fda3de-ca8f-4f01-824c-68fe0c03c177 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:44.072378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fba3a1-faed-4b60-9732-f8fd10ceb139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:44.073086 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fba3a1-faed-4b60-9732-f8fd10ceb139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:44.073364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fba3a1-faed-4b60-9732-f8fd10ceb139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:44.073637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fba3a1-faed-4b60-9732-f8fd10ceb139 admin admin] PATH_INFO: `/v3/users/8636ca57428f420d9572df6af5ae0233` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:44.074789 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fba3a1-faed-4b60-9732-f8fd10ceb139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8636ca57428f420d9572df6af5ae0233)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:44.077168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fba3a1-faed-4b60-9732-f8fd10ceb139 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:44.290369 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:44.318437 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b3fba3a1-faed-4b60-9732-f8fd10ceb139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8636ca57428f420d9572df6af5ae0233'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:44.319377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b3fba3a1-faed-4b60-9732-f8fd10ceb139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8636ca57428f420d9572df6af5ae0233'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:44.352181 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b3fba3a1-faed-4b60-9732-f8fd10ceb139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8636ca57428f420d9572df6af5ae0233'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:44.849066 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b3fba3a1-faed-4b60-9732-f8fd10ceb139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8636ca57428f420d9572df6af5ae0233'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:44.849686 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b3fba3a1-faed-4b60-9732-f8fd10ceb139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8636ca57428f420d9572df6af5ae0233'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:44.862684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1032/2090] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:43 2022] DELETE /identity/v3/users/8636ca57428f420d9572df6af5ae0233 => generated 0 bytes in 987 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:15:45.176725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc825f79-2aa8-4197-b0ff-7e96a3af9878 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:45.178243 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dc825f79-2aa8-4197-b0ff-7e96a3af9878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:45.198491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc825f79-2aa8-4197-b0ff-7e96a3af9878 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:45.290980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53589d-9ae7-4327-86cb-aa5866329c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:45.292587 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53589d-9ae7-4327-86cb-aa5866329c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:45.292870 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53589d-9ae7-4327-86cb-aa5866329c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:45.293177 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53589d-9ae7-4327-86cb-aa5866329c7f admin admin] PATH_INFO: `/v3/projects/1f05f015bcb84862b4c298c5a2fc5526` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:45.294526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53589d-9ae7-4327-86cb-aa5866329c7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f05f015bcb84862b4c298c5a2fc5526)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:45.296612 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53589d-9ae7-4327-86cb-aa5866329c7f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:45.429016 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd53589d-9ae7-4327-86cb-aa5866329c7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f05f015bcb84862b4c298c5a2fc5526', 'request_id': 'req-dd53589d-9ae7-4327-86cb-aa5866329c7f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:45.438297 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd53589d-9ae7-4327-86cb-aa5866329c7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f05f015bcb84862b4c298c5a2fc5526', 'request_id': 'req-dd53589d-9ae7-4327-86cb-aa5866329c7f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:45.506202 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd53589d-9ae7-4327-86cb-aa5866329c7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f05f015bcb84862b4c298c5a2fc5526', 'request_id': 'req-dd53589d-9ae7-4327-86cb-aa5866329c7f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:45.531951 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd53589d-9ae7-4327-86cb-aa5866329c7f admin admin] The token cache is being invalidate because project 1f05f015bcb84862b4c298c5a2fc5526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:45.532272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd53589d-9ae7-4327-86cb-aa5866329c7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:45.536026 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd53589d-9ae7-4327-86cb-aa5866329c7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:45.540823 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1059/2091] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:45 2022] DELETE /identity/v3/projects/1f05f015bcb84862b4c298c5a2fc5526 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:45.544322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b30d4-db15-49b2-ba74-eae4ce45f4ae None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:45.544851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b30d4-db15-49b2-ba74-eae4ce45f4ae None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:45.545007 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b30d4-db15-49b2-ba74-eae4ce45f4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:45.584240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-0e3b30d4-db15-49b2-ba74-eae4ce45f4ae None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:45.586126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0e3b30d4-db15-49b2-ba74-eae4ce45f4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:45.728619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1033/2092] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:45.731434 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc4437-8a51-41eb-9f47-bec2dbf26d68 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:45.731774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc4437-8a51-41eb-9f47-bec2dbf26d68 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:45.734675 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc4437-8a51-41eb-9f47-bec2dbf26d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:45.762082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0edc4437-8a51-41eb-9f47-bec2dbf26d68 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:45.763530 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0edc4437-8a51-41eb-9f47-bec2dbf26d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:45.774982 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1060/2093] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:45.777337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b30d4-db15-49b2-ba74-eae4ce45f4ae None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:45.791126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b30d4-db15-49b2-ba74-eae4ce45f4ae None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:45.797612 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746823a7-a74b-49ad-8a27-816458dd44bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:45.798500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746823a7-a74b-49ad-8a27-816458dd44bc admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:45.798777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746823a7-a74b-49ad-8a27-816458dd44bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:45.799305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746823a7-a74b-49ad-8a27-816458dd44bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:45.806352 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746823a7-a74b-49ad-8a27-816458dd44bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:15:45.806352 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746823a7-a74b-49ad-8a27-816458dd44bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:45.806352 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746823a7-a74b-49ad-8a27-816458dd44bc admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:45.820623 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1034/2094] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:15:45.825513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc4437-8a51-41eb-9f47-bec2dbf26d68 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:45.836022 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc4437-8a51-41eb-9f47-bec2dbf26d68 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:45.841665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501eb68-3a1f-4327-80c1-c2dbceb96767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:45.842701 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501eb68-3a1f-4327-80c1-c2dbceb96767 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:45.843131 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501eb68-3a1f-4327-80c1-c2dbceb96767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:45.843669 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501eb68-3a1f-4327-80c1-c2dbceb96767 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:45.844887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501eb68-3a1f-4327-80c1-c2dbceb96767 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:45.847806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501eb68-3a1f-4327-80c1-c2dbceb96767 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:46.070560 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1061/2095] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:45 2022] POST /identity/v3/projects => generated 385 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:15:46.071444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746823a7-a74b-49ad-8a27-816458dd44bc admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:46.087177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746823a7-a74b-49ad-8a27-816458dd44bc admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:46.151859 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501eb68-3a1f-4327-80c1-c2dbceb96767 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:46.156166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3501eb68-3a1f-4327-80c1-c2dbceb96767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:46.161419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34722a-0f05-4460-b08d-56ab255364e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:46.162178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34722a-0f05-4460-b08d-56ab255364e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:46.163156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34722a-0f05-4460-b08d-56ab255364e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:46.167830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34722a-0f05-4460-b08d-56ab255364e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:46.168728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34722a-0f05-4460-b08d-56ab255364e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:46.171722 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34722a-0f05-4460-b08d-56ab255364e3 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:46.195429 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501eb68-3a1f-4327-80c1-c2dbceb96767 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:46.200324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c6db8-aacc-4c92-a764-966faba36614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:46.200957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c6db8-aacc-4c92-a764-966faba36614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:46.201246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c6db8-aacc-4c92-a764-966faba36614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:46.201496 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c6db8-aacc-4c92-a764-966faba36614 admin admin] PATH_INFO: `/v3/projects/6e5a598e3aac46438eca22fcbd9f223c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:46.205338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c6db8-aacc-4c92-a764-966faba36614 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e5a598e3aac46438eca22fcbd9f223c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:46.205338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c6db8-aacc-4c92-a764-966faba36614 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:46.222722 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:46.258594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1035/2096] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:46 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:15:46.259951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34722a-0f05-4460-b08d-56ab255364e3 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:46.286679 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34722a-0f05-4460-b08d-56ab255364e3 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:46.292742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45081504-f0f9-4506-ba09-d50a75f2877f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:46.293554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45081504-f0f9-4506-ba09-d50a75f2877f admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:46.293901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45081504-f0f9-4506-ba09-d50a75f2877f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:46.294302 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45081504-f0f9-4506-ba09-d50a75f2877f admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:46.295114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45081504-f0f9-4506-ba09-d50a75f2877f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:46.297757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45081504-f0f9-4506-ba09-d50a75f2877f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:46.313145 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1036/2097] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:15:46.316211 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45081504-f0f9-4506-ba09-d50a75f2877f admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:46.328797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45081504-f0f9-4506-ba09-d50a75f2877f admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:46.342597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-653c6db8-aacc-4c92-a764-966faba36614 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e5a598e3aac46438eca22fcbd9f223c', 'request_id': 'req-653c6db8-aacc-4c92-a764-966faba36614'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:46.347866 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:46.350215 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-653c6db8-aacc-4c92-a764-966faba36614 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e5a598e3aac46438eca22fcbd9f223c', 'request_id': 'req-653c6db8-aacc-4c92-a764-966faba36614'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:46.397593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202a3e4-6752-4ab9-b036-f103616ffc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:46.398196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202a3e4-6752-4ab9-b036-f103616ffc65 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:46.398470 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202a3e4-6752-4ab9-b036-f103616ffc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:46.398759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202a3e4-6752-4ab9-b036-f103616ffc65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:46.399501 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202a3e4-6752-4ab9-b036-f103616ffc65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:46.401166 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202a3e4-6752-4ab9-b036-f103616ffc65 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:46.414349 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1037/2098] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:46.415646 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202a3e4-6752-4ab9-b036-f103616ffc65 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:46.426103 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202a3e4-6752-4ab9-b036-f103616ffc65 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:46.430819 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc2002-8158-4364-88d6-ac469b6e8bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:46.432236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc2002-8158-4364-88d6-ac469b6e8bfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:46.432516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc2002-8158-4364-88d6-ac469b6e8bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:46.432785 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc2002-8158-4364-88d6-ac469b6e8bfd admin admin] PATH_INFO: `/v3/projects/6a11f2c5d4444338bb1fec447a4abbae/users/feb0d4bd46a148adbe69d7e284823759/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:46.447945 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc2002-8158-4364-88d6-ac469b6e8bfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a11f2c5d4444338bb1fec447a4abbae, user_id=feb0d4bd46a148adbe69d7e284823759, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:46.451026 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc2002-8158-4364-88d6-ac469b6e8bfd admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:46.479887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-653c6db8-aacc-4c92-a764-966faba36614 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e5a598e3aac46438eca22fcbd9f223c', 'request_id': 'req-653c6db8-aacc-4c92-a764-966faba36614'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:46.483781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-653c6db8-aacc-4c92-a764-966faba36614 admin admin] The token cache is being invalidate because project 6e5a598e3aac46438eca22fcbd9f223c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:46.484073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-653c6db8-aacc-4c92-a764-966faba36614 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:46.484737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-653c6db8-aacc-4c92-a764-966faba36614 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:46.486230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1062/2099] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:46 2022] DELETE /identity/v3/projects/6e5a598e3aac46438eca22fcbd9f223c => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:15:46.491625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eab544-ec6c-4b98-a87d-d8fd8f148c93 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:46.492346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eab544-ec6c-4b98-a87d-d8fd8f148c93 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:46.492623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eab544-ec6c-4b98-a87d-d8fd8f148c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:46.515138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-66eab544-ec6c-4b98-a87d-d8fd8f148c93 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:46.516325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-66eab544-ec6c-4b98-a87d-d8fd8f148c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:46.524368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1063/2100] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:15:46.534076 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a817018-7981-4f25-8b58-667eb3b508dc None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:46.534356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a817018-7981-4f25-8b58-667eb3b508dc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:46.534624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a817018-7981-4f25-8b58-667eb3b508dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:46.554846 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7a817018-7981-4f25-8b58-667eb3b508dc None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:46.556321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7a817018-7981-4f25-8b58-667eb3b508dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:46.564473 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1038/2101] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:46 2022] PUT /identity/v3/projects/6a11f2c5d4444338bb1fec447a4abbae/users/feb0d4bd46a148adbe69d7e284823759/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:15:46.570963 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd536c06-8f80-490a-b8a7-be0f682393e1 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:46.571425 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd536c06-8f80-490a-b8a7-be0f682393e1 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:46.571701 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd536c06-8f80-490a-b8a7-be0f682393e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:46.591854 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-bd536c06-8f80-490a-b8a7-be0f682393e1 None None] MFA Rules not processed for user `e767c7081875428382af11d1ff475a20`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:46.593060 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-bd536c06-8f80-490a-b8a7-be0f682393e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:46.623905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1064/2102] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:46.625213 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce195d8-c8d3-4122-b4cd-6525fd998207 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:46.625492 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce195d8-c8d3-4122-b4cd-6525fd998207 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:46.625757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce195d8-c8d3-4122-b4cd-6525fd998207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:46.653335 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fce195d8-c8d3-4122-b4cd-6525fd998207 None None] MFA Rules not processed for user `feb0d4bd46a148adbe69d7e284823759`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:46.653908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1039/2103] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:46.658231 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fce195d8-c8d3-4122-b4cd-6525fd998207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:46.658368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd536c06-8f80-490a-b8a7-be0f682393e1 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:46.668827 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd536c06-8f80-490a-b8a7-be0f682393e1 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:46.673708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b91f0-b0bb-4255-9480-c9cdc37b130a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:46.674444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b91f0-b0bb-4255-9480-c9cdc37b130a admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:46.674725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b91f0-b0bb-4255-9480-c9cdc37b130a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:46.675271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b91f0-b0bb-4255-9480-c9cdc37b130a admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:46.675791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b91f0-b0bb-4255-9480-c9cdc37b130a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:15:46.676093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b91f0-b0bb-4255-9480-c9cdc37b130a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:46.677847 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b91f0-b0bb-4255-9480-c9cdc37b130a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:46.695185 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1040/2104] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:15:46.697346 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b91f0-b0bb-4255-9480-c9cdc37b130a admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:46.708298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b91f0-b0bb-4255-9480-c9cdc37b130a admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:46.713850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4531c3-fc76-41db-b31e-846cc19c5e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:46.714595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4531c3-fc76-41db-b31e-846cc19c5e66 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:46.715060 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4531c3-fc76-41db-b31e-846cc19c5e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:46.715161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4531c3-fc76-41db-b31e-846cc19c5e66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:46.715822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4531c3-fc76-41db-b31e-846cc19c5e66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:46.717860 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4531c3-fc76-41db-b31e-846cc19c5e66 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:46.735651 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1065/2105] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:46.739788 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce195d8-c8d3-4122-b4cd-6525fd998207 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:46.741280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fce195d8-c8d3-4122-b4cd-6525fd998207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:46.755162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce195d8-c8d3-4122-b4cd-6525fd998207 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:46.760451 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9f4d5-a141-476e-858a-ef7cc53557f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:46.761212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9f4d5-a141-476e-858a-ef7cc53557f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:46.761473 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9f4d5-a141-476e-858a-ef7cc53557f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:46.761744 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9f4d5-a141-476e-858a-ef7cc53557f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:46.762400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9f4d5-a141-476e-858a-ef7cc53557f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:46.764332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9f4d5-a141-476e-858a-ef7cc53557f5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:46.802252 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1041/2106] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:46 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:46.803298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4531c3-fc76-41db-b31e-846cc19c5e66 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:46.875779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1066/2107] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:46 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:15:46.879875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9f4d5-a141-476e-858a-ef7cc53557f5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:46.890909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:46.907275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4531c3-fc76-41db-b31e-846cc19c5e66 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:46.911398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:46.962494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9f4d5-a141-476e-858a-ef7cc53557f5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:46.968129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfb4f3-9bda-4e75-900b-82d69ce48644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:46.968792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfb4f3-9bda-4e75-900b-82d69ce48644 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:46.969068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfb4f3-9bda-4e75-900b-82d69ce48644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:46.969339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfb4f3-9bda-4e75-900b-82d69ce48644 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:46.970005 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfb4f3-9bda-4e75-900b-82d69ce48644 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:46.971970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfb4f3-9bda-4e75-900b-82d69ce48644 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:46.973114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4f3aa-bd28-4552-94c2-e2c9b82777f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:46.974020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4f3aa-bd28-4552-94c2-e2c9b82777f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:46.974473 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4f3aa-bd28-4552-94c2-e2c9b82777f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:46.975165 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4f3aa-bd28-4552-94c2-e2c9b82777f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:46.976113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4f3aa-bd28-4552-94c2-e2c9b82777f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:46.978494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4f3aa-bd28-4552-94c2-e2c9b82777f0 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:47.090895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1067/2108] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:46 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:15:47.095155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfb4f3-9bda-4e75-900b-82d69ce48644 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:47.114116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfb4f3-9bda-4e75-900b-82d69ce48644 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:47.124793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1042/2109] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:46 2022] POST /identity/v3/users => generated 292 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:15:47.125993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4f3aa-bd28-4552-94c2-e2c9b82777f0 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:47.136256 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0da02-aa61-4c19-9eb0-122d7a9204bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:47.136972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0da02-aa61-4c19-9eb0-122d7a9204bf admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:47.137358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0da02-aa61-4c19-9eb0-122d7a9204bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:47.137702 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0da02-aa61-4c19-9eb0-122d7a9204bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:47.138442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0da02-aa61-4c19-9eb0-122d7a9204bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:47.142308 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4f3aa-bd28-4552-94c2-e2c9b82777f0 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:47.150916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133ea9b-9bd9-4b36-bcc4-58803f5221f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:47.152943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133ea9b-9bd9-4b36-bcc4-58803f5221f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:47.154803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133ea9b-9bd9-4b36-bcc4-58803f5221f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:47.154803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133ea9b-9bd9-4b36-bcc4-58803f5221f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:47.154803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133ea9b-9bd9-4b36-bcc4-58803f5221f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:47.157424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133ea9b-9bd9-4b36-bcc4-58803f5221f7 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:47.175093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1043/2110] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:15:47.178625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133ea9b-9bd9-4b36-bcc4-58803f5221f7 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:47.187588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0da02-aa61-4c19-9eb0-122d7a9204bf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:47.195751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133ea9b-9bd9-4b36-bcc4-58803f5221f7 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:47.203442 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f759a-081a-4a7c-a7f0-e5a870f0fc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:47.204122 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1068/2111] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:15:47.205918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f759a-081a-4a7c-a7f0-e5a870f0fc76 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:47.205918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f759a-081a-4a7c-a7f0-e5a870f0fc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:47.205918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f759a-081a-4a7c-a7f0-e5a870f0fc76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:47.206372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f759a-081a-4a7c-a7f0-e5a870f0fc76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:47.209195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f759a-081a-4a7c-a7f0-e5a870f0fc76 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:47.212396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0da02-aa61-4c19-9eb0-122d7a9204bf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:47.229567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1044/2112] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:15:47.230696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0da02-aa61-4c19-9eb0-122d7a9204bf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:47.234202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f759a-081a-4a7c-a7f0-e5a870f0fc76 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:47.237988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e95be8-4fbd-4d26-ab0b-26cafdc3a65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:47.239426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e95be8-4fbd-4d26-ab0b-26cafdc3a65f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:47.239426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e95be8-4fbd-4d26-ab0b-26cafdc3a65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:47.239426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e95be8-4fbd-4d26-ab0b-26cafdc3a65f admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:47.240316 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e95be8-4fbd-4d26-ab0b-26cafdc3a65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:47.241969 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e95be8-4fbd-4d26-ab0b-26cafdc3a65f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:47.254799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1069/2113] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:47.269268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e95be8-4fbd-4d26-ab0b-26cafdc3a65f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:47.269372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f759a-081a-4a7c-a7f0-e5a870f0fc76 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:47.274679 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72431bd7-1bc7-475e-8cf0-8c977dbf2cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:47.275249 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e95be8-4fbd-4d26-ab0b-26cafdc3a65f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:47.275432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72431bd7-1bc7-475e-8cf0-8c977dbf2cf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:47.275799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72431bd7-1bc7-475e-8cf0-8c977dbf2cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:47.275799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72431bd7-1bc7-475e-8cf0-8c977dbf2cf7 admin admin] PATH_INFO: `/v3/projects/12ab2bfef5bb4e67b5103759904c6a9f/users/4a95ac03c42e4ffe9243c5e52ef8a4cf/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:47.281567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4999fff-0481-4427-8da0-df23c636a6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:47.282328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4999fff-0481-4427-8da0-df23c636a6c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:47.282700 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4999fff-0481-4427-8da0-df23c636a6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:47.283122 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4999fff-0481-4427-8da0-df23c636a6c5 admin admin] PATH_INFO: `/v3/projects/2e0ecfb2fc454932b646f4f15c98c903/users/972e72cc6a8441d5beb8ce3a1b6278ee/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:47.299790 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4999fff-0481-4427-8da0-df23c636a6c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e0ecfb2fc454932b646f4f15c98c903, user_id=972e72cc6a8441d5beb8ce3a1b6278ee, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:47.302614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4999fff-0481-4427-8da0-df23c636a6c5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:47.308300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72431bd7-1bc7-475e-8cf0-8c977dbf2cf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12ab2bfef5bb4e67b5103759904c6a9f, user_id=4a95ac03c42e4ffe9243c5e52ef8a4cf, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:47.311507 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72431bd7-1bc7-475e-8cf0-8c977dbf2cf7 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:47.408924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1070/2114] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:47 2022] PUT /identity/v3/projects/2e0ecfb2fc454932b646f4f15c98c903/users/972e72cc6a8441d5beb8ce3a1b6278ee/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:15:47.419605 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad6e6d-604c-4d97-b143-665edd84684a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:47.419910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad6e6d-604c-4d97-b143-665edd84684a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:47.420179 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad6e6d-604c-4d97-b143-665edd84684a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:47.438029 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1045/2115] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:47 2022] PUT /identity/v3/projects/12ab2bfef5bb4e67b5103759904c6a9f/users/4a95ac03c42e4ffe9243c5e52ef8a4cf/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:47.447952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad93b0-b3f5-4759-99cc-575d8765a795 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:47.448253 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad93b0-b3f5-4759-99cc-575d8765a795 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:47.448609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad93b0-b3f5-4759-99cc-575d8765a795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:47.453256 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-58ad6e6d-604c-4d97-b143-665edd84684a None None] MFA Rules not processed for user `972e72cc6a8441d5beb8ce3a1b6278ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:47.454499 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-58ad6e6d-604c-4d97-b143-665edd84684a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:47.499068 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-11ad93b0-b3f5-4759-99cc-575d8765a795 None None] MFA Rules not processed for user `4a95ac03c42e4ffe9243c5e52ef8a4cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:47.501466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-11ad93b0-b3f5-4759-99cc-575d8765a795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:47.564363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1071/2116] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:15:47.569121 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad6e6d-604c-4d97-b143-665edd84684a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:47.593517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1046/2117] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:15:47.612854 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad93b0-b3f5-4759-99cc-575d8765a795 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:47.616975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-11ad93b0-b3f5-4759-99cc-575d8765a795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:47.647073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:47.680884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad6e6d-604c-4d97-b143-665edd84684a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:47.691332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c688faf-27b9-4aff-8f7c-19262391be1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:47.692095 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c688faf-27b9-4aff-8f7c-19262391be1c admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:47.692985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c688faf-27b9-4aff-8f7c-19262391be1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:47.693791 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c688faf-27b9-4aff-8f7c-19262391be1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:47.694679 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c688faf-27b9-4aff-8f7c-19262391be1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:47.699799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:47.700017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c688faf-27b9-4aff-8f7c-19262391be1c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:47.714697 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad93b0-b3f5-4759-99cc-575d8765a795 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:47.718670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1d62a-eb46-4f52-805f-cac5aae7cf8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:47.719413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1d62a-eb46-4f52-805f-cac5aae7cf8b service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:47.720239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1d62a-eb46-4f52-805f-cac5aae7cf8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:47.720522 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1d62a-eb46-4f52-805f-cac5aae7cf8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:47.730557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1d62a-eb46-4f52-805f-cac5aae7cf8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:47.732215 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1d62a-eb46-4f52-805f-cac5aae7cf8b service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:47.742142 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1047/2118] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:47.768283 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1d62a-eb46-4f52-805f-cac5aae7cf8b service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:47.780019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1d62a-eb46-4f52-805f-cac5aae7cf8b service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:47.784830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb09be-98b3-42a5-9bde-1a220c89bd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:47.785444 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb09be-98b3-42a5-9bde-1a220c89bd17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:47.785734 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb09be-98b3-42a5-9bde-1a220c89bd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:47.785992 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb09be-98b3-42a5-9bde-1a220c89bd17 admin admin] PATH_INFO: `/v3/users/4a95ac03c42e4ffe9243c5e52ef8a4cf` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:47.787186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb09be-98b3-42a5-9bde-1a220c89bd17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a95ac03c42e4ffe9243c5e52ef8a4cf)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:47.789121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb09be-98b3-42a5-9bde-1a220c89bd17 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:47.846418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1072/2119] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:47 2022] POST /identity/v3/projects => generated 385 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:47.850342 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c688faf-27b9-4aff-8f7c-19262391be1c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:47.912647 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c688faf-27b9-4aff-8f7c-19262391be1c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:47.917534 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e6fb6-550a-421d-8cac-50ff4f09a23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:47.918127 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e6fb6-550a-421d-8cac-50ff4f09a23c admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:47.918400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e6fb6-550a-421d-8cac-50ff4f09a23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:47.918669 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e6fb6-550a-421d-8cac-50ff4f09a23c admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:47.919339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e6fb6-550a-421d-8cac-50ff4f09a23c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:47.921247 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e6fb6-550a-421d-8cac-50ff4f09a23c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:48.055223 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1073/2120] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:47 2022] POST /identity/v3/users => generated 299 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:15:48.056278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e6fb6-550a-421d-8cac-50ff4f09a23c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:48.065380 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b5cb09be-98b3-42a5-9bde-1a220c89bd17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a95ac03c42e4ffe9243c5e52ef8a4cf'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:48.066142 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b5cb09be-98b3-42a5-9bde-1a220c89bd17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a95ac03c42e4ffe9243c5e52ef8a4cf'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:48.069704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e6fb6-550a-421d-8cac-50ff4f09a23c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:48.077364 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2b33b-b595-4ada-bddd-2536d15913e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:48.078537 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2b33b-b595-4ada-bddd-2536d15913e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:48.079223 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2b33b-b595-4ada-bddd-2536d15913e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:48.079769 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2b33b-b595-4ada-bddd-2536d15913e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:48.080176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b5cb09be-98b3-42a5-9bde-1a220c89bd17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a95ac03c42e4ffe9243c5e52ef8a4cf'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:48.080475 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2b33b-b595-4ada-bddd-2536d15913e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:48.082137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2b33b-b595-4ada-bddd-2536d15913e6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:48.092578 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1074/2121] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:48.096782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2b33b-b595-4ada-bddd-2536d15913e6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:48.098720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-61b2b33b-b595-4ada-bddd-2536d15913e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:48.111361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2b33b-b595-4ada-bddd-2536d15913e6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:48.115871 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8bb9c-87ce-4ce3-9e90-76590d0df621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:48.116490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8bb9c-87ce-4ce3-9e90-76590d0df621 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:48.116761 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8bb9c-87ce-4ce3-9e90-76590d0df621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:48.117025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8bb9c-87ce-4ce3-9e90-76590d0df621 admin admin] PATH_INFO: `/v3/projects/74d6395d52bd43498013426ae7e9d00c/users/ec2c3ec9582846bd9f50e1780ab7e377/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:48.133085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c8bb9c-87ce-4ce3-9e90-76590d0df621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d6395d52bd43498013426ae7e9d00c, user_id=ec2c3ec9582846bd9f50e1780ab7e377, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:48.135935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c8bb9c-87ce-4ce3-9e90-76590d0df621 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:48.250675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b5cb09be-98b3-42a5-9bde-1a220c89bd17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a95ac03c42e4ffe9243c5e52ef8a4cf'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:48.251558 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b5cb09be-98b3-42a5-9bde-1a220c89bd17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a95ac03c42e4ffe9243c5e52ef8a4cf'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:48.256011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1075/2122] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:48 2022] PUT /identity/v3/projects/74d6395d52bd43498013426ae7e9d00c/users/ec2c3ec9582846bd9f50e1780ab7e377/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:15:48.260249 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8bb9c-87ce-4ce3-9e90-76590d0df621 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:48.267594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1048/2123] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:47 2022] DELETE /identity/v3/users/4a95ac03c42e4ffe9243c5e52ef8a4cf => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:48.272222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96643413-da53-4c06-84c6-4624d0329fa9 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:48.272612 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96643413-da53-4c06-84c6-4624d0329fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:48.272825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96643413-da53-4c06-84c6-4624d0329fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:48.296797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-96643413-da53-4c06-84c6-4624d0329fa9 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:48.298030 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-96643413-da53-4c06-84c6-4624d0329fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:48.331353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8bb9c-87ce-4ce3-9e90-76590d0df621 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:48.360685 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:48.362254 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1049/2124] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:15:48.372166 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96643413-da53-4c06-84c6-4624d0329fa9 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:48.374492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-96643413-da53-4c06-84c6-4624d0329fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:48.409148 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03c1a9-10b1-4e92-80c4-f4f15660c710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:48.409868 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03c1a9-10b1-4e92-80c4-f4f15660c710 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:48.410144 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03c1a9-10b1-4e92-80c4-f4f15660c710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:48.410420 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03c1a9-10b1-4e92-80c4-f4f15660c710 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:48.411072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03c1a9-10b1-4e92-80c4-f4f15660c710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:48.412658 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03c1a9-10b1-4e92-80c4-f4f15660c710 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:48.424993 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1076/2125] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:15:48.429163 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03c1a9-10b1-4e92-80c4-f4f15660c710 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:48.441201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03c1a9-10b1-4e92-80c4-f4f15660c710 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:48.446168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01602c9b-16be-4e5e-b5a8-5eeee05f5cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:48.448160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01602c9b-16be-4e5e-b5a8-5eeee05f5cea admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:48.448518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01602c9b-16be-4e5e-b5a8-5eeee05f5cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:48.448888 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01602c9b-16be-4e5e-b5a8-5eeee05f5cea admin admin] PATH_INFO: `/v3/projects/74d6395d52bd43498013426ae7e9d00c/users/ec2c3ec9582846bd9f50e1780ab7e377/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:48.451772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01602c9b-16be-4e5e-b5a8-5eeee05f5cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d6395d52bd43498013426ae7e9d00c, user_id=ec2c3ec9582846bd9f50e1780ab7e377, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:48.460031 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01602c9b-16be-4e5e-b5a8-5eeee05f5cea admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:48.466616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96643413-da53-4c06-84c6-4624d0329fa9 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:48.474077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1077/2126] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:48 2022] PUT /identity/v3/projects/74d6395d52bd43498013426ae7e9d00c/users/ec2c3ec9582846bd9f50e1780ab7e377/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:48.488298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:48.488484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5e1d0-5880-4f94-a7ca-5a6bf9317afe None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:48.488585 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5e1d0-5880-4f94-a7ca-5a6bf9317afe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:48.488840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5e1d0-5880-4f94-a7ca-5a6bf9317afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:48.518078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e4a5e1d0-5880-4f94-a7ca-5a6bf9317afe None None] MFA Rules not processed for user `ec2c3ec9582846bd9f50e1780ab7e377`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:48.519326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e4a5e1d0-5880-4f94-a7ca-5a6bf9317afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:48.533324 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0748c-332e-498d-9814-b9af778d6ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:48.533928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0748c-332e-498d-9814-b9af778d6ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:48.534202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0748c-332e-498d-9814-b9af778d6ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:48.534468 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0748c-332e-498d-9814-b9af778d6ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:48.607093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0748c-332e-498d-9814-b9af778d6ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:48.608691 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0748c-332e-498d-9814-b9af778d6ea4 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:48.611650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1078/2127] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:15:48.620576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1050/2128] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:15:48.625009 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5e1d0-5880-4f94-a7ca-5a6bf9317afe None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:48.627540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e4a5e1d0-5880-4f94-a7ca-5a6bf9317afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:48.687144 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5e1d0-5880-4f94-a7ca-5a6bf9317afe None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:48.690794 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0204feb2-36b7-4b9e-964c-b69f6ee34656 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:48.691427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204feb2-36b7-4b9e-964c-b69f6ee34656 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:48.691696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204feb2-36b7-4b9e-964c-b69f6ee34656 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:48.691962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204feb2-36b7-4b9e-964c-b69f6ee34656 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:48.693703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0204feb2-36b7-4b9e-964c-b69f6ee34656 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:48.754601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0204feb2-36b7-4b9e-964c-b69f6ee34656 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:48.756073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0204feb2-36b7-4b9e-964c-b69f6ee34656 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:48.766114 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1079/2129] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:48.879367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:49.124379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0748c-332e-498d-9814-b9af778d6ea4 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:49.125952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-abc0748c-332e-498d-9814-b9af778d6ea4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:49.148535 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0748c-332e-498d-9814-b9af778d6ea4 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:49.154374 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb7463-2c6e-4717-bd24-355431f7e1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:49.155133 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb7463-2c6e-4717-bd24-355431f7e1dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:49.155310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb7463-2c6e-4717-bd24-355431f7e1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:49.155568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb7463-2c6e-4717-bd24-355431f7e1dc admin admin] PATH_INFO: `/v3/projects/12ab2bfef5bb4e67b5103759904c6a9f` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:49.157120 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb7463-2c6e-4717-bd24-355431f7e1dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=12ab2bfef5bb4e67b5103759904c6a9f)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:49.158840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb7463-2c6e-4717-bd24-355431f7e1dc admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:49.332301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c9eb7463-2c6e-4717-bd24-355431f7e1dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12ab2bfef5bb4e67b5103759904c6a9f', 'request_id': 'req-c9eb7463-2c6e-4717-bd24-355431f7e1dc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:49.339739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c9eb7463-2c6e-4717-bd24-355431f7e1dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12ab2bfef5bb4e67b5103759904c6a9f', 'request_id': 'req-c9eb7463-2c6e-4717-bd24-355431f7e1dc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:49.466917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c9eb7463-2c6e-4717-bd24-355431f7e1dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12ab2bfef5bb4e67b5103759904c6a9f', 'request_id': 'req-c9eb7463-2c6e-4717-bd24-355431f7e1dc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:49.477825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c9eb7463-2c6e-4717-bd24-355431f7e1dc admin admin] The token cache is being invalidate because project 12ab2bfef5bb4e67b5103759904c6a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:49.478147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c9eb7463-2c6e-4717-bd24-355431f7e1dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:49.478781 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c9eb7463-2c6e-4717-bd24-355431f7e1dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:49.480451 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1051/2130] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:49 2022] DELETE /identity/v3/projects/12ab2bfef5bb4e67b5103759904c6a9f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:49.487446 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91544c-7229-4499-b9ed-309b3c118bc2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:49.488284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91544c-7229-4499-b9ed-309b3c118bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:49.488738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91544c-7229-4499-b9ed-309b3c118bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:49.509614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fb91544c-7229-4499-b9ed-309b3c118bc2 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:49.511683 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fb91544c-7229-4499-b9ed-309b3c118bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:49.582815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1080/2131] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:49.585774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862747f-efd1-4dc7-ba95-739beeb56c2d None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:49.585774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862747f-efd1-4dc7-ba95-739beeb56c2d None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:49.585947 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862747f-efd1-4dc7-ba95-739beeb56c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:49.609522 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-b862747f-efd1-4dc7-ba95-739beeb56c2d None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:49.610966 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b862747f-efd1-4dc7-ba95-739beeb56c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:49.620433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1052/2132] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:15:49.631145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91544c-7229-4499-b9ed-309b3c118bc2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:49.644941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91544c-7229-4499-b9ed-309b3c118bc2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:49.655383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc725ba-7a32-4ac6-b346-6f4434e48b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:49.655383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc725ba-7a32-4ac6-b346-6f4434e48b14 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:49.655383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc725ba-7a32-4ac6-b346-6f4434e48b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:49.655383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc725ba-7a32-4ac6-b346-6f4434e48b14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:49.655808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc725ba-7a32-4ac6-b346-6f4434e48b14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:15:49.655808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc725ba-7a32-4ac6-b346-6f4434e48b14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:49.669521 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc725ba-7a32-4ac6-b346-6f4434e48b14 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:49.692733 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1081/2133] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:15:49.694415 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862747f-efd1-4dc7-ba95-739beeb56c2d None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:49.719040 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862747f-efd1-4dc7-ba95-739beeb56c2d None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:49.726054 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc115454-32f4-4dcd-bca1-a9cf9a9750cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:49.727053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc115454-32f4-4dcd-bca1-a9cf9a9750cb admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:49.730730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc115454-32f4-4dcd-bca1-a9cf9a9750cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:49.730730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc115454-32f4-4dcd-bca1-a9cf9a9750cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:49.730730 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc115454-32f4-4dcd-bca1-a9cf9a9750cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:49.731893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc115454-32f4-4dcd-bca1-a9cf9a9750cb admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:49.851338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1053/2134] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:49 2022] POST /identity/v3/projects => generated 397 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:49.856666 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc725ba-7a32-4ac6-b346-6f4434e48b14 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:49.866998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc725ba-7a32-4ac6-b346-6f4434e48b14 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:49.928348 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e290702-c7b3-4611-b1f6-456c03425679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:49.928876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e290702-c7b3-4611-b1f6-456c03425679 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:49.929149 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e290702-c7b3-4611-b1f6-456c03425679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:49.929416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e290702-c7b3-4611-b1f6-456c03425679 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:49.930044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e290702-c7b3-4611-b1f6-456c03425679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:49.933173 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e290702-c7b3-4611-b1f6-456c03425679 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:50.038059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1082/2135] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:49 2022] POST /identity/v3/users => generated 299 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:50.043649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc115454-32f4-4dcd-bca1-a9cf9a9750cb admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:50.054178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc115454-32f4-4dcd-bca1-a9cf9a9750cb admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:50.060838 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f995a-fa89-498e-80e6-1d74397344f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:50.062005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f995a-fa89-498e-80e6-1d74397344f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:50.062005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f995a-fa89-498e-80e6-1d74397344f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:50.062005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f995a-fa89-498e-80e6-1d74397344f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:50.062661 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f995a-fa89-498e-80e6-1d74397344f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:50.064356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f995a-fa89-498e-80e6-1d74397344f6 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:50.078083 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1054/2136] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:15:50.079296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e290702-c7b3-4611-b1f6-456c03425679 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:50.091051 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e290702-c7b3-4611-b1f6-456c03425679 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:50.096050 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af86f9-7a07-439b-9dd3-fef0a2babf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:50.096738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af86f9-7a07-439b-9dd3-fef0a2babf46 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:50.097127 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af86f9-7a07-439b-9dd3-fef0a2babf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:50.097402 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af86f9-7a07-439b-9dd3-fef0a2babf46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:50.098043 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af86f9-7a07-439b-9dd3-fef0a2babf46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:50.099949 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af86f9-7a07-439b-9dd3-fef0a2babf46 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:50.114294 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1083/2137] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:50.115704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f995a-fa89-498e-80e6-1d74397344f6 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:50.123304 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:50.129618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f995a-fa89-498e-80e6-1d74397344f6 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:50.134360 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba924176-813a-44f8-94b0-6ae25815475f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:50.136826 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba924176-813a-44f8-94b0-6ae25815475f admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:50.136826 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba924176-813a-44f8-94b0-6ae25815475f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:50.136826 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba924176-813a-44f8-94b0-6ae25815475f admin admin] PATH_INFO: `/v3/projects/bf9ab67aaefd4e92896635481cc6705f/users/0c09ef7ae3bd4db8aa32601c55b168cc/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:50.155740 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba924176-813a-44f8-94b0-6ae25815475f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf9ab67aaefd4e92896635481cc6705f, user_id=0c09ef7ae3bd4db8aa32601c55b168cc, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:50.158423 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba924176-813a-44f8-94b0-6ae25815475f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:50.266912 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1055/2138] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:50 2022] PUT /identity/v3/projects/bf9ab67aaefd4e92896635481cc6705f/users/0c09ef7ae3bd4db8aa32601c55b168cc/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:50.278134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966c8b7-8929-4714-8b47-a45d3e3916aa None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:50.278415 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966c8b7-8929-4714-8b47-a45d3e3916aa None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:50.278683 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966c8b7-8929-4714-8b47-a45d3e3916aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:50.308614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e966c8b7-8929-4714-8b47-a45d3e3916aa None None] MFA Rules not processed for user `0c09ef7ae3bd4db8aa32601c55b168cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:50.309862 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e966c8b7-8929-4714-8b47-a45d3e3916aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:50.354596 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:50.375627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1084/2139] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:50 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:50.387853 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba924176-813a-44f8-94b0-6ae25815475f admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:50.389627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-ba924176-813a-44f8-94b0-6ae25815475f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:50.449071 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba924176-813a-44f8-94b0-6ae25815475f admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:50.452708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f66834-d869-4647-81f3-b559768d752d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:50.453339 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f66834-d869-4647-81f3-b559768d752d service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:50.453613 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f66834-d869-4647-81f3-b559768d752d service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:50.453877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f66834-d869-4647-81f3-b559768d752d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:50.466628 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f66834-d869-4647-81f3-b559768d752d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:50.468095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f66834-d869-4647-81f3-b559768d752d service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:50.477043 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1056/2140] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:50 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:51.868201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966c8b7-8929-4714-8b47-a45d3e3916aa None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:51.871540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e966c8b7-8929-4714-8b47-a45d3e3916aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:51.953854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966c8b7-8929-4714-8b47-a45d3e3916aa None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:51.958511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2406b35-a6cd-48f3-a000-295aaefb5286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:51.959170 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2406b35-a6cd-48f3-a000-295aaefb5286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:51.959444 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2406b35-a6cd-48f3-a000-295aaefb5286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:51.959709 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2406b35-a6cd-48f3-a000-295aaefb5286 admin admin] PATH_INFO: `/v3/users/1a968944f4bc4c5b93f98d124b88fe01` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:51.962856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2406b35-a6cd-48f3-a000-295aaefb5286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a968944f4bc4c5b93f98d124b88fe01)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:51.964988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2406b35-a6cd-48f3-a000-295aaefb5286 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:52.107962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f66834-d869-4647-81f3-b559768d752d service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:52.109442 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-40f66834-d869-4647-81f3-b559768d752d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:52.121957 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f66834-d869-4647-81f3-b559768d752d service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:52.128532 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5159c3d-5ea5-46d7-8311-e686af268135 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:52.130919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159c3d-5ea5-46d7-8311-e686af268135 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:52.131110 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159c3d-5ea5-46d7-8311-e686af268135 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:52.132228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159c3d-5ea5-46d7-8311-e686af268135 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:52.133633 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e5159c3d-5ea5-46d7-8311-e686af268135 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:52.145092 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b2406b35-a6cd-48f3-a000-295aaefb5286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a968944f4bc4c5b93f98d124b88fe01'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:52.168754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b2406b35-a6cd-48f3-a000-295aaefb5286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a968944f4bc4c5b93f98d124b88fe01'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:52.169505 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b2406b35-a6cd-48f3-a000-295aaefb5286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a968944f4bc4c5b93f98d124b88fe01'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:52.211720 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5159c3d-5ea5-46d7-8311-e686af268135 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:52.214337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5159c3d-5ea5-46d7-8311-e686af268135 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:52.223969 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1057/2141] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:15:52.255563 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b2406b35-a6cd-48f3-a000-295aaefb5286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a968944f4bc4c5b93f98d124b88fe01'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:52.256266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b2406b35-a6cd-48f3-a000-295aaefb5286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a968944f4bc4c5b93f98d124b88fe01'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:52.269701 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1085/2142] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:51 2022] DELETE /identity/v3/users/1a968944f4bc4c5b93f98d124b88fe01 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:15:52.274298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bd481-4796-4798-b359-b48ff9d32b4a None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:52.274516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bd481-4796-4798-b359-b48ff9d32b4a None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:52.274793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bd481-4796-4798-b359-b48ff9d32b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:52.293934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-086bd481-4796-4798-b359-b48ff9d32b4a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:52.295167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-086bd481-4796-4798-b359-b48ff9d32b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:52.356212 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1058/2143] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:15:52.365910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2406b35-a6cd-48f3-a000-295aaefb5286 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:52.367476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b2406b35-a6cd-48f3-a000-295aaefb5286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:52.425215 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2406b35-a6cd-48f3-a000-295aaefb5286 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:52.428979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34d76e-660d-494d-a9ea-4d054c143655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:52.429588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34d76e-660d-494d-a9ea-4d054c143655 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:52.429863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34d76e-660d-494d-a9ea-4d054c143655 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:52.430167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34d76e-660d-494d-a9ea-4d054c143655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:52.440129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34d76e-660d-494d-a9ea-4d054c143655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:52.441558 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34d76e-660d-494d-a9ea-4d054c143655 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:52.452562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1086/2144] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:52.795765 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23fe95-8d28-4b7e-bd82-731906756aa7 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:52.795765 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23fe95-8d28-4b7e-bd82-731906756aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:52.795765 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23fe95-8d28-4b7e-bd82-731906756aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:52.816248 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-2f23fe95-8d28-4b7e-bd82-731906756aa7 None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:52.817504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-2f23fe95-8d28-4b7e-bd82-731906756aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:52.837725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:52.903175 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1059/2145] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Oct 6 08:15:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:52.918668 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34d76e-660d-494d-a9ea-4d054c143655 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:52.920232 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2d34d76e-660d-494d-a9ea-4d054c143655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:52.937101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:52.992499 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34d76e-660d-494d-a9ea-4d054c143655 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:52.998626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c29ce4-5178-468a-a748-c34620916b50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:52.999262 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c29ce4-5178-468a-a748-c34620916b50 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:52.999531 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c29ce4-5178-468a-a748-c34620916b50 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:52.999794 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c29ce4-5178-468a-a748-c34620916b50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:53.008383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c29ce4-5178-468a-a748-c34620916b50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:53.010597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c29ce4-5178-468a-a748-c34620916b50 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:53.019877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1087/2146] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:15:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:15:53.050968 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23fe95-8d28-4b7e-bd82-731906756aa7 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:53.052535 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-2f23fe95-8d28-4b7e-bd82-731906756aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:53.068504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23fe95-8d28-4b7e-bd82-731906756aa7 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:53.074471 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00748a-ac0d-45bb-a0c0-ea0546d0bfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:53.074762 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00748a-ac0d-45bb-a0c0-ea0546d0bfe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:53.075139 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00748a-ac0d-45bb-a0c0-ea0546d0bfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:53.075473 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00748a-ac0d-45bb-a0c0-ea0546d0bfe7 admin admin] PATH_INFO: `/v3/projects/6822b951e5b445a09a05693b1b1cf209` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:53.076815 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00748a-ac0d-45bb-a0c0-ea0546d0bfe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6822b951e5b445a09a05693b1b1cf209)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:53.078862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00748a-ac0d-45bb-a0c0-ea0546d0bfe7 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:53.183860 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9c00748a-ac0d-45bb-a0c0-ea0546d0bfe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6822b951e5b445a09a05693b1b1cf209', 'request_id': 'req-9c00748a-ac0d-45bb-a0c0-ea0546d0bfe7'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:53.196577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9c00748a-ac0d-45bb-a0c0-ea0546d0bfe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6822b951e5b445a09a05693b1b1cf209', 'request_id': 'req-9c00748a-ac0d-45bb-a0c0-ea0546d0bfe7'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:53.329308 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9c00748a-ac0d-45bb-a0c0-ea0546d0bfe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6822b951e5b445a09a05693b1b1cf209', 'request_id': 'req-9c00748a-ac0d-45bb-a0c0-ea0546d0bfe7'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:53.347647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9c00748a-ac0d-45bb-a0c0-ea0546d0bfe7 admin admin] The token cache is being invalidate because project 6822b951e5b445a09a05693b1b1cf209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:53.348224 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9c00748a-ac0d-45bb-a0c0-ea0546d0bfe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:53.355482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9c00748a-ac0d-45bb-a0c0-ea0546d0bfe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:53.357852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1060/2147] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:53 2022] DELETE /identity/v3/projects/6822b951e5b445a09a05693b1b1cf209 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:53.362974 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde0c5b-8577-4a2e-a574-a7eeb3f84855 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:53.363275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde0c5b-8577-4a2e-a574-a7eeb3f84855 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:53.363661 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde0c5b-8577-4a2e-a574-a7eeb3f84855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:53.403491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2fde0c5b-8577-4a2e-a574-a7eeb3f84855 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:53.405221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2fde0c5b-8577-4a2e-a574-a7eeb3f84855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:53.507597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1088/2148] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:53.508555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13f584-d708-4aad-83cd-40a40e36eca4 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:53.508930 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13f584-d708-4aad-83cd-40a40e36eca4 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:53.509268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13f584-d708-4aad-83cd-40a40e36eca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:53.549552 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-9f13f584-d708-4aad-83cd-40a40e36eca4 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:53.550604 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9f13f584-d708-4aad-83cd-40a40e36eca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:53.563447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1061/2149] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:15:53.569583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde0c5b-8577-4a2e-a574-a7eeb3f84855 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:53.582561 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde0c5b-8577-4a2e-a574-a7eeb3f84855 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:53.590250 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b9013-53a9-485c-bcfd-953e26759b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:53.591319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b9013-53a9-485c-bcfd-953e26759b3f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:53.591756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b9013-53a9-485c-bcfd-953e26759b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:53.592151 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b9013-53a9-485c-bcfd-953e26759b3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:53.593324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b9013-53a9-485c-bcfd-953e26759b3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:15:53.593869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b9013-53a9-485c-bcfd-953e26759b3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:53.596106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b9013-53a9-485c-bcfd-953e26759b3f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:53.617152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1089/2150] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:15:53.622174 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13f584-d708-4aad-83cd-40a40e36eca4 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:53.634093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13f584-d708-4aad-83cd-40a40e36eca4 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:53.639876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad9b13-66a2-4566-a4ea-0abcd9967e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:53.641741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad9b13-66a2-4566-a4ea-0abcd9967e14 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:53.642022 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad9b13-66a2-4566-a4ea-0abcd9967e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:53.642649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad9b13-66a2-4566-a4ea-0abcd9967e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:53.643734 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad9b13-66a2-4566-a4ea-0abcd9967e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:53.646643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad9b13-66a2-4566-a4ea-0abcd9967e14 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:53.754575 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1062/2151] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:53 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:53.759373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b9013-53a9-485c-bcfd-953e26759b3f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:53.770438 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b9013-53a9-485c-bcfd-953e26759b3f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:53.923933 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff005b-106a-4f27-89d5-9df8c6d5d9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:53.928204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff005b-106a-4f27-89d5-9df8c6d5d9e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:53.928204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff005b-106a-4f27-89d5-9df8c6d5d9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:53.928204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff005b-106a-4f27-89d5-9df8c6d5d9e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:53.928204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff005b-106a-4f27-89d5-9df8c6d5d9e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:53.928204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff005b-106a-4f27-89d5-9df8c6d5d9e7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:54.027937 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad9b13-66a2-4566-a4ea-0abcd9967e14 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:54.029719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-38ad9b13-66a2-4566-a4ea-0abcd9967e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:54.085212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1090/2152] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:53 2022] POST /identity/v3/users => generated 286 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:54.090770 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff005b-106a-4f27-89d5-9df8c6d5d9e7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:54.108995 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff005b-106a-4f27-89d5-9df8c6d5d9e7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:54.113755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a8d23-f93e-453e-abd0-c7b9b217bfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:54.114355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a8d23-f93e-453e-abd0-c7b9b217bfdc admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:54.114630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a8d23-f93e-453e-abd0-c7b9b217bfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:54.114906 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a8d23-f93e-453e-abd0-c7b9b217bfdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:54.115596 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a8d23-f93e-453e-abd0-c7b9b217bfdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:54.117274 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a8d23-f93e-453e-abd0-c7b9b217bfdc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:54.119385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad9b13-66a2-4566-a4ea-0abcd9967e14 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:54.125087 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278015a0-48b2-4b68-8318-a3eb8c6928a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:54.126136 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278015a0-48b2-4b68-8318-a3eb8c6928a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:54.126689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278015a0-48b2-4b68-8318-a3eb8c6928a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:54.127530 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278015a0-48b2-4b68-8318-a3eb8c6928a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:54.130586 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-278015a0-48b2-4b68-8318-a3eb8c6928a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:54.155018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1091/2153] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:54.165024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a8d23-f93e-453e-abd0-c7b9b217bfdc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:54.181946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a8d23-f93e-453e-abd0-c7b9b217bfdc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:54.182310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:54.187044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b50c9-ab2a-4174-aa69-68d22273633a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:54.187646 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b50c9-ab2a-4174-aa69-68d22273633a admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:54.187931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b50c9-ab2a-4174-aa69-68d22273633a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:54.188309 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b50c9-ab2a-4174-aa69-68d22273633a admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:54.188951 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b50c9-ab2a-4174-aa69-68d22273633a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:54.190563 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b50c9-ab2a-4174-aa69-68d22273633a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:54.201554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1092/2154] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:15:54.205984 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b50c9-ab2a-4174-aa69-68d22273633a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:54.217216 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278015a0-48b2-4b68-8318-a3eb8c6928a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:54.218671 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278015a0-48b2-4b68-8318-a3eb8c6928a9 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:54.220462 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b50c9-ab2a-4174-aa69-68d22273633a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:54.226081 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fb279-b33a-48a6-9561-cd9e0592e990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:54.227264 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fb279-b33a-48a6-9561-cd9e0592e990 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:54.227709 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fb279-b33a-48a6-9561-cd9e0592e990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:54.228344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fb279-b33a-48a6-9561-cd9e0592e990 admin admin] PATH_INFO: `/v3/projects/2395ed6f9e954e989b96cd7e824cc54f/users/5d521ad6b5534800ac02e05b15c7a6e7/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:54.242460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1063/2155] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:54.259244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fb279-b33a-48a6-9561-cd9e0592e990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2395ed6f9e954e989b96cd7e824cc54f, user_id=5d521ad6b5534800ac02e05b15c7a6e7, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:54.264258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fb279-b33a-48a6-9561-cd9e0592e990 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:54.335539 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1093/2156] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:54 2022] PUT /identity/v3/projects/2395ed6f9e954e989b96cd7e824cc54f/users/5d521ad6b5534800ac02e05b15c7a6e7/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:15:54.348276 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec80d3-8502-48a5-b0b2-53abc9fcdd36 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:54.348463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec80d3-8502-48a5-b0b2-53abc9fcdd36 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:54.348950 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec80d3-8502-48a5-b0b2-53abc9fcdd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:54.381675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-beec80d3-8502-48a5-b0b2-53abc9fcdd36 None None] MFA Rules not processed for user `5d521ad6b5534800ac02e05b15c7a6e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:54.382925 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-beec80d3-8502-48a5-b0b2-53abc9fcdd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:54.481230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1064/2157] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:54.489389 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:54.493705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fb279-b33a-48a6-9561-cd9e0592e990 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:54.576056 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fb279-b33a-48a6-9561-cd9e0592e990 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:54.580071 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5578ca-e8ea-46b2-ac6e-aae82a857fe6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:54.581362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5578ca-e8ea-46b2-ac6e-aae82a857fe6 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:54.581362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5578ca-e8ea-46b2-ac6e-aae82a857fe6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:54.581548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5578ca-e8ea-46b2-ac6e-aae82a857fe6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:54.592764 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5578ca-e8ea-46b2-ac6e-aae82a857fe6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:54.594308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5578ca-e8ea-46b2-ac6e-aae82a857fe6 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:54.604711 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1094/2158] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:54.637909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec80d3-8502-48a5-b0b2-53abc9fcdd36 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:54.707272 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec80d3-8502-48a5-b0b2-53abc9fcdd36 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:54.712344 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39737d-46be-4ff5-ae44-6207cb931cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:54.713197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39737d-46be-4ff5-ae44-6207cb931cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:54.713472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39737d-46be-4ff5-ae44-6207cb931cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:54.713746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39737d-46be-4ff5-ae44-6207cb931cf5 admin admin] PATH_INFO: `/v3/users/5d521ad6b5534800ac02e05b15c7a6e7` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:54.715016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39737d-46be-4ff5-ae44-6207cb931cf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d521ad6b5534800ac02e05b15c7a6e7)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:54.716981 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39737d-46be-4ff5-ae44-6207cb931cf5 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:54.936540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bc39737d-46be-4ff5-ae44-6207cb931cf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d521ad6b5534800ac02e05b15c7a6e7'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:54.937199 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bc39737d-46be-4ff5-ae44-6207cb931cf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d521ad6b5534800ac02e05b15c7a6e7'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:54.949680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bc39737d-46be-4ff5-ae44-6207cb931cf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d521ad6b5534800ac02e05b15c7a6e7'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:55.003054 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bc39737d-46be-4ff5-ae44-6207cb931cf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d521ad6b5534800ac02e05b15c7a6e7'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:55.003672 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bc39737d-46be-4ff5-ae44-6207cb931cf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d521ad6b5534800ac02e05b15c7a6e7'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:55.016792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1065/2159] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:15:54 2022] DELETE /identity/v3/users/5d521ad6b5534800ac02e05b15c7a6e7 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:55.021288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc926f-3f02-4f20-b959-c5779dfd41e7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:55.021762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc926f-3f02-4f20-b959-c5779dfd41e7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:55.021889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc926f-3f02-4f20-b959-c5779dfd41e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:55.041358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-74bc926f-3f02-4f20-b959-c5779dfd41e7 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:55.042687 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-74bc926f-3f02-4f20-b959-c5779dfd41e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:55.089528 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:55.108893 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1095/2160] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:55.118460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39737d-46be-4ff5-ae44-6207cb931cf5 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:55.121372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-bc39737d-46be-4ff5-ae44-6207cb931cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:55.192910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39737d-46be-4ff5-ae44-6207cb931cf5 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:55.196658 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3528a8-f5ca-4a16-ad0c-1970004a094c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:55.197307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3528a8-f5ca-4a16-ad0c-1970004a094c service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:55.197577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3528a8-f5ca-4a16-ad0c-1970004a094c service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:55.197841 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3528a8-f5ca-4a16-ad0c-1970004a094c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:55.211846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3528a8-f5ca-4a16-ad0c-1970004a094c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:55.213900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3528a8-f5ca-4a16-ad0c-1970004a094c service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:55.224446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1066/2161] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:55.688116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc926f-3f02-4f20-b959-c5779dfd41e7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:55.689544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-74bc926f-3f02-4f20-b959-c5779dfd41e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:55.703650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc926f-3f02-4f20-b959-c5779dfd41e7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:55.708570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa21662-24a9-4ff1-af83-ee2c69d28be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:55.709700 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa21662-24a9-4ff1-af83-ee2c69d28be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:55.709973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa21662-24a9-4ff1-af83-ee2c69d28be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:55.710265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa21662-24a9-4ff1-af83-ee2c69d28be0 admin admin] PATH_INFO: `/v3/projects/2395ed6f9e954e989b96cd7e824cc54f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:55.711485 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa21662-24a9-4ff1-af83-ee2c69d28be0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2395ed6f9e954e989b96cd7e824cc54f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:55.713666 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa21662-24a9-4ff1-af83-ee2c69d28be0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:55.870474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8fa21662-24a9-4ff1-af83-ee2c69d28be0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2395ed6f9e954e989b96cd7e824cc54f', 'request_id': 'req-8fa21662-24a9-4ff1-af83-ee2c69d28be0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:55.877597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8fa21662-24a9-4ff1-af83-ee2c69d28be0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2395ed6f9e954e989b96cd7e824cc54f', 'request_id': 'req-8fa21662-24a9-4ff1-af83-ee2c69d28be0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:55.951995 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8fa21662-24a9-4ff1-af83-ee2c69d28be0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2395ed6f9e954e989b96cd7e824cc54f', 'request_id': 'req-8fa21662-24a9-4ff1-af83-ee2c69d28be0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:55.956743 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:55.968343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8fa21662-24a9-4ff1-af83-ee2c69d28be0 admin admin] The token cache is being invalidate because project 2395ed6f9e954e989b96cd7e824cc54f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:15:55.968964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8fa21662-24a9-4ff1-af83-ee2c69d28be0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:55.969927 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8fa21662-24a9-4ff1-af83-ee2c69d28be0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:15:55.972212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1096/2162] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:15:55 2022] DELETE /identity/v3/projects/2395ed6f9e954e989b96cd7e824cc54f => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:55.977004 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adc526-524e-442f-b994-121d46ad6fef None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:55.977700 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adc526-524e-442f-b994-121d46ad6fef None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:55.977982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adc526-524e-442f-b994-121d46ad6fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:56.015236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-88adc526-524e-442f-b994-121d46ad6fef None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:56.016952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-88adc526-524e-442f-b994-121d46ad6fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:56.090207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1067/2163] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:56.097838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18050356-06e1-4089-8156-35b4ed5b1fa7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:56.098369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18050356-06e1-4089-8156-35b4ed5b1fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:56.098369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18050356-06e1-4089-8156-35b4ed5b1fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:56.119329 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-18050356-06e1-4089-8156-35b4ed5b1fa7 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:56.120792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-18050356-06e1-4089-8156-35b4ed5b1fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:56.128958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1097/2164] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:56.133541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88adc526-524e-442f-b994-121d46ad6fef None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:56.143356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88adc526-524e-442f-b994-121d46ad6fef None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:56.148158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1c40b-9c2b-4de3-9a0d-6913ff280ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:56.148760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1c40b-9c2b-4de3-9a0d-6913ff280ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:56.149156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1c40b-9c2b-4de3-9a0d-6913ff280ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:56.149428 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1c40b-9c2b-4de3-9a0d-6913ff280ee5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:56.150633 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1c40b-9c2b-4de3-9a0d-6913ff280ee5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:15:56.150950 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1c40b-9c2b-4de3-9a0d-6913ff280ee5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:56.152717 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1c40b-9c2b-4de3-9a0d-6913ff280ee5 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:56.164701 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1068/2165] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:15:56.169677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18050356-06e1-4089-8156-35b4ed5b1fa7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:56.178676 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18050356-06e1-4089-8156-35b4ed5b1fa7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:56.183595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593938b-e04a-4253-a420-952bbbdfe9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:56.184175 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593938b-e04a-4253-a420-952bbbdfe9f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:56.184444 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593938b-e04a-4253-a420-952bbbdfe9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:56.184708 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593938b-e04a-4253-a420-952bbbdfe9f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:56.185326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3593938b-e04a-4253-a420-952bbbdfe9f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:56.187265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3593938b-e04a-4253-a420-952bbbdfe9f3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:56.290143 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1098/2166] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:15:56 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:56.294383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1c40b-9c2b-4de3-9a0d-6913ff280ee5 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:56.304666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1c40b-9c2b-4de3-9a0d-6913ff280ee5 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:56.360765 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523a8f2-257b-40bf-b5b1-e439e637c36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:56.361364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523a8f2-257b-40bf-b5b1-e439e637c36e admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:56.361643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523a8f2-257b-40bf-b5b1-e439e637c36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:56.361938 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523a8f2-257b-40bf-b5b1-e439e637c36e admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:56.362571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523a8f2-257b-40bf-b5b1-e439e637c36e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:56.364529 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523a8f2-257b-40bf-b5b1-e439e637c36e admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:56.467099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:15:56.472286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1069/2167] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:15:56 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:15:56.476435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593938b-e04a-4253-a420-952bbbdfe9f3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:56.487469 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593938b-e04a-4253-a420-952bbbdfe9f3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:56.492056 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594a55f-ee38-484f-8009-e49e45391708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:56.492741 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594a55f-ee38-484f-8009-e49e45391708 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:56.493019 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594a55f-ee38-484f-8009-e49e45391708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:56.493287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594a55f-ee38-484f-8009-e49e45391708 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:56.494701 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594a55f-ee38-484f-8009-e49e45391708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:56.496542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594a55f-ee38-484f-8009-e49e45391708 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:56.506537 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1099/2168] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:15:56.510572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523a8f2-257b-40bf-b5b1-e439e637c36e admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:56.520631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523a8f2-257b-40bf-b5b1-e439e637c36e admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:56.525592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24702a04-20bf-455b-ae03-ca93fe5d7023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:56.526230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24702a04-20bf-455b-ae03-ca93fe5d7023 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:56.526544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24702a04-20bf-455b-ae03-ca93fe5d7023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:56.526838 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24702a04-20bf-455b-ae03-ca93fe5d7023 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:56.527571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24702a04-20bf-455b-ae03-ca93fe5d7023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:56.529171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24702a04-20bf-455b-ae03-ca93fe5d7023 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:56.539760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1070/2169] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:15:56.544405 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594a55f-ee38-484f-8009-e49e45391708 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:56.554116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594a55f-ee38-484f-8009-e49e45391708 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:56.559711 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474413c-5285-4f07-b58e-1ed956ebc03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:56.560349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474413c-5285-4f07-b58e-1ed956ebc03a admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:56.560624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474413c-5285-4f07-b58e-1ed956ebc03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:56.560894 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474413c-5285-4f07-b58e-1ed956ebc03a admin admin] PATH_INFO: `/v3/projects/141863eb21404f229b8d624af2ff6743/users/fff5a88cba04475c9a0142156a2269f0/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:56.575629 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474413c-5285-4f07-b58e-1ed956ebc03a admin admin] RBAC: Authorizing `identity:create_grant(project_id=141863eb21404f229b8d624af2ff6743, user_id=fff5a88cba04475c9a0142156a2269f0, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:56.579304 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474413c-5285-4f07-b58e-1ed956ebc03a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:56.694739 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1100/2170] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:15:56 2022] PUT /identity/v3/projects/141863eb21404f229b8d624af2ff6743/users/fff5a88cba04475c9a0142156a2269f0/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:15:56.700019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ab48a-1ad2-4a53-b0a6-058fa80cc16b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:56.700551 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ab48a-1ad2-4a53-b0a6-058fa80cc16b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:56.701009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ab48a-1ad2-4a53-b0a6-058fa80cc16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:56.720979 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-3c9ab48a-1ad2-4a53-b0a6-058fa80cc16b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:56.722176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3c9ab48a-1ad2-4a53-b0a6-058fa80cc16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:56.781241 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1071/2171] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:15:56.795351 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474413c-5285-4f07-b58e-1ed956ebc03a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:56.797008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7474413c-5285-4f07-b58e-1ed956ebc03a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:56.857771 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474413c-5285-4f07-b58e-1ed956ebc03a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:56.861566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2eba7f-464f-4db7-8ab2-cc946a2a2316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:56.862172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2eba7f-464f-4db7-8ab2-cc946a2a2316 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:56.862454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2eba7f-464f-4db7-8ab2-cc946a2a2316 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:56.862753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2eba7f-464f-4db7-8ab2-cc946a2a2316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:56.878885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2eba7f-464f-4db7-8ab2-cc946a2a2316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:56.881068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2eba7f-464f-4db7-8ab2-cc946a2a2316 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:56.894662 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1101/2172] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:15:59.206561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8283c-9848-4bc0-af7b-af71ed91c348 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:59.207476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8283c-9848-4bc0-af7b-af71ed91c348 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:59.207476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8283c-9848-4bc0-af7b-af71ed91c348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:59.239507 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-b7f8283c-9848-4bc0-af7b-af71ed91c348 None None] MFA Rules not processed for user `fff5a88cba04475c9a0142156a2269f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:15:59.241132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b7f8283c-9848-4bc0-af7b-af71ed91c348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:59.352384 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1072/2173] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:15:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:15:59.360705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2eba7f-464f-4db7-8ab2-cc946a2a2316 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:15:59.362236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cb2eba7f-464f-4db7-8ab2-cc946a2a2316 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:15:59.464933 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2eba7f-464f-4db7-8ab2-cc946a2a2316 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:15:59.471470 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b224e5-3fe0-47d6-859a-a1807f22ad0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:15:59.471934 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b224e5-3fe0-47d6-859a-a1807f22ad0a service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:15:59.472208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b224e5-3fe0-47d6-859a-a1807f22ad0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:15:59.472477 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b224e5-3fe0-47d6-859a-a1807f22ad0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:15:59.483858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b224e5-3fe0-47d6-859a-a1807f22ad0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:15:59.488963 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b224e5-3fe0-47d6-859a-a1807f22ad0a service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:15:59.521463 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1102/2174] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:15:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:16:03.344793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f5b7b-bea5-4295-a601-f2d7f2e312e0 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:03.345508 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f5b7b-bea5-4295-a601-f2d7f2e312e0 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:03.345508 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f5b7b-bea5-4295-a601-f2d7f2e312e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:03.365883 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-280f5b7b-bea5-4295-a601-f2d7f2e312e0 None None] MFA Rules not processed for user `fff5a88cba04475c9a0142156a2269f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:16:03.367287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-280f5b7b-bea5-4295-a601-f2d7f2e312e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:03.375208 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1073/2175] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:16:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:16:05.788319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b224e5-3fe0-47d6-859a-a1807f22ad0a service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:05.790385 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-94b224e5-3fe0-47d6-859a-a1807f22ad0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:05.808421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b224e5-3fe0-47d6-859a-a1807f22ad0a service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:05.813126 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f3917-797d-4961-8818-c3723916ca66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:05.813847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f3917-797d-4961-8818-c3723916ca66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:05.814198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f3917-797d-4961-8818-c3723916ca66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:05.814530 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f3917-797d-4961-8818-c3723916ca66 admin admin] PATH_INFO: `/v3/users/0c09ef7ae3bd4db8aa32601c55b168cc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:05.815779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f3917-797d-4961-8818-c3723916ca66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c09ef7ae3bd4db8aa32601c55b168cc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:05.817804 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f3917-797d-4961-8818-c3723916ca66 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:05.957796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:16:05.976578 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7d1f3917-797d-4961-8818-c3723916ca66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c09ef7ae3bd4db8aa32601c55b168cc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:05.994668 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7d1f3917-797d-4961-8818-c3723916ca66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c09ef7ae3bd4db8aa32601c55b168cc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:05.995428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7d1f3917-797d-4961-8818-c3723916ca66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c09ef7ae3bd4db8aa32601c55b168cc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:06.045828 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7d1f3917-797d-4961-8818-c3723916ca66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c09ef7ae3bd4db8aa32601c55b168cc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:06.046617 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7d1f3917-797d-4961-8818-c3723916ca66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c09ef7ae3bd4db8aa32601c55b168cc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:06.060559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1103/2176] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:16:05 2022] DELETE /identity/v3/users/0c09ef7ae3bd4db8aa32601c55b168cc => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:16:06.065671 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4794e-fab9-4131-9d21-d1e4efecde4b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:06.066205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4794e-fab9-4131-9d21-d1e4efecde4b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:06.066294 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4794e-fab9-4131-9d21-d1e4efecde4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:06.086749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-c8e4794e-fab9-4131-9d21-d1e4efecde4b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:16:06.088817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-c8e4794e-fab9-4131-9d21-d1e4efecde4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:06.165277 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1074/2177] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:16:06.175475 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f3917-797d-4961-8818-c3723916ca66 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:06.176872 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7d1f3917-797d-4961-8818-c3723916ca66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:06.247038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f3917-797d-4961-8818-c3723916ca66 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:06.253663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723462a3-ad5e-46bd-9130-c860639184cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:06.253663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723462a3-ad5e-46bd-9130-c860639184cd service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:06.253663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723462a3-ad5e-46bd-9130-c860639184cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:06.253663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723462a3-ad5e-46bd-9130-c860639184cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:06.262441 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723462a3-ad5e-46bd-9130-c860639184cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:06.263938 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723462a3-ad5e-46bd-9130-c860639184cd service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:06.280667 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1104/2178] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:16:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:16:06.707418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4794e-fab9-4131-9d21-d1e4efecde4b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:06.712193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-c8e4794e-fab9-4131-9d21-d1e4efecde4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:06.721837 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4794e-fab9-4131-9d21-d1e4efecde4b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:06.726733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708dc811-4c59-4eae-b9e8-6dd330f7412b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:06.727607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708dc811-4c59-4eae-b9e8-6dd330f7412b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:06.727947 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708dc811-4c59-4eae-b9e8-6dd330f7412b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:06.728342 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708dc811-4c59-4eae-b9e8-6dd330f7412b admin admin] PATH_INFO: `/v3/projects/bf9ab67aaefd4e92896635481cc6705f` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:06.729627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708dc811-4c59-4eae-b9e8-6dd330f7412b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf9ab67aaefd4e92896635481cc6705f)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:06.731569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708dc811-4c59-4eae-b9e8-6dd330f7412b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:06.740083 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:16:07.094523 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-708dc811-4c59-4eae-b9e8-6dd330f7412b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf9ab67aaefd4e92896635481cc6705f', 'request_id': 'req-708dc811-4c59-4eae-b9e8-6dd330f7412b'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:07.100592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-708dc811-4c59-4eae-b9e8-6dd330f7412b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf9ab67aaefd4e92896635481cc6705f', 'request_id': 'req-708dc811-4c59-4eae-b9e8-6dd330f7412b'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:07.174292 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-708dc811-4c59-4eae-b9e8-6dd330f7412b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf9ab67aaefd4e92896635481cc6705f', 'request_id': 'req-708dc811-4c59-4eae-b9e8-6dd330f7412b'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:07.187184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-708dc811-4c59-4eae-b9e8-6dd330f7412b admin admin] The token cache is being invalidate because project bf9ab67aaefd4e92896635481cc6705f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:16:07.187184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-708dc811-4c59-4eae-b9e8-6dd330f7412b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:07.187557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-708dc811-4c59-4eae-b9e8-6dd330f7412b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:07.189050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1075/2179] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:16:06 2022] DELETE /identity/v3/projects/bf9ab67aaefd4e92896635481cc6705f => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:16:08.779746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723462a3-ad5e-46bd-9130-c860639184cd service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:08.781470 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-723462a3-ad5e-46bd-9130-c860639184cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:08.796388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723462a3-ad5e-46bd-9130-c860639184cd service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:08.864930 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a69da-3058-43cd-8e04-0a82166af674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:08.865557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a69da-3058-43cd-8e04-0a82166af674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:08.865821 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a69da-3058-43cd-8e04-0a82166af674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:08.866091 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a69da-3058-43cd-8e04-0a82166af674 admin admin] PATH_INFO: `/v3/users/feb0d4bd46a148adbe69d7e284823759` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:08.867432 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a69da-3058-43cd-8e04-0a82166af674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb0d4bd46a148adbe69d7e284823759)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:08.870216 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a69da-3058-43cd-8e04-0a82166af674 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:09.090526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4d8a69da-3058-43cd-8e04-0a82166af674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb0d4bd46a148adbe69d7e284823759'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:09.095196 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:16:09.103750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4d8a69da-3058-43cd-8e04-0a82166af674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb0d4bd46a148adbe69d7e284823759'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:09.104655 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4d8a69da-3058-43cd-8e04-0a82166af674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb0d4bd46a148adbe69d7e284823759'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:09.177631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4d8a69da-3058-43cd-8e04-0a82166af674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb0d4bd46a148adbe69d7e284823759'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:09.178295 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4d8a69da-3058-43cd-8e04-0a82166af674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb0d4bd46a148adbe69d7e284823759'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:09.195438 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1105/2180] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:16:08 2022] DELETE /identity/v3/users/feb0d4bd46a148adbe69d7e284823759 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:16:09.200984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d81632-2883-444e-8ef3-e7b545aaee26 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:09.201371 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d81632-2883-444e-8ef3-e7b545aaee26 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:09.201690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d81632-2883-444e-8ef3-e7b545aaee26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:09.232274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-b9d81632-2883-444e-8ef3-e7b545aaee26 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:16:09.234324 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b9d81632-2883-444e-8ef3-e7b545aaee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:09.312978 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1076/2181] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:16:09.325536 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a69da-3058-43cd-8e04-0a82166af674 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:09.327703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4d8a69da-3058-43cd-8e04-0a82166af674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:09.398934 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a69da-3058-43cd-8e04-0a82166af674 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:09.403594 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c720e-2b26-468a-95f4-bf84344d050d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:09.404197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c720e-2b26-468a-95f4-bf84344d050d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:09.404471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c720e-2b26-468a-95f4-bf84344d050d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:09.404733 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c720e-2b26-468a-95f4-bf84344d050d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:09.418517 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c720e-2b26-468a-95f4-bf84344d050d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:09.419987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c720e-2b26-468a-95f4-bf84344d050d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:09.429283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1106/2182] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:16:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:16:09.868720 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d81632-2883-444e-8ef3-e7b545aaee26 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:09.872003 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b9d81632-2883-444e-8ef3-e7b545aaee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:09.903268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d81632-2883-444e-8ef3-e7b545aaee26 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:09.906199 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102098ee-71bd-429f-ba94-93b917e9bdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:09.906705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102098ee-71bd-429f-ba94-93b917e9bdd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:09.907036 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102098ee-71bd-429f-ba94-93b917e9bdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:09.907310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102098ee-71bd-429f-ba94-93b917e9bdd5 admin admin] PATH_INFO: `/v3/projects/6a11f2c5d4444338bb1fec447a4abbae` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:09.908459 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102098ee-71bd-429f-ba94-93b917e9bdd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a11f2c5d4444338bb1fec447a4abbae)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:09.910448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102098ee-71bd-429f-ba94-93b917e9bdd5 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:10.020584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-102098ee-71bd-429f-ba94-93b917e9bdd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a11f2c5d4444338bb1fec447a4abbae', 'request_id': 'req-102098ee-71bd-429f-ba94-93b917e9bdd5'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:10.028529 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-102098ee-71bd-429f-ba94-93b917e9bdd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a11f2c5d4444338bb1fec447a4abbae', 'request_id': 'req-102098ee-71bd-429f-ba94-93b917e9bdd5'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:10.420729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-102098ee-71bd-429f-ba94-93b917e9bdd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a11f2c5d4444338bb1fec447a4abbae', 'request_id': 'req-102098ee-71bd-429f-ba94-93b917e9bdd5'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:10.434803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-102098ee-71bd-429f-ba94-93b917e9bdd5 admin admin] The token cache is being invalidate because project 6a11f2c5d4444338bb1fec447a4abbae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:16:10.435381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-102098ee-71bd-429f-ba94-93b917e9bdd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:10.439378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-102098ee-71bd-429f-ba94-93b917e9bdd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:10.441368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1077/2183] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:16:09 2022] DELETE /identity/v3/projects/6a11f2c5d4444338bb1fec447a4abbae => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:16:10.446825 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c720e-2b26-468a-95f4-bf84344d050d service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:10.458055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c720e-2b26-468a-95f4-bf84344d050d service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:10.542584 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e57b9-a78d-4ba2-a5b3-de0fe5280620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:10.543256 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e57b9-a78d-4ba2-a5b3-de0fe5280620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:10.543527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e57b9-a78d-4ba2-a5b3-de0fe5280620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:10.544290 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e57b9-a78d-4ba2-a5b3-de0fe5280620 admin admin] PATH_INFO: `/v3/users/972e72cc6a8441d5beb8ce3a1b6278ee` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:10.544972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e57b9-a78d-4ba2-a5b3-de0fe5280620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=972e72cc6a8441d5beb8ce3a1b6278ee)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:10.547074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e57b9-a78d-4ba2-a5b3-de0fe5280620 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:10.767263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1b7e57b9-a78d-4ba2-a5b3-de0fe5280620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '972e72cc6a8441d5beb8ce3a1b6278ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:10.781791 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1b7e57b9-a78d-4ba2-a5b3-de0fe5280620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '972e72cc6a8441d5beb8ce3a1b6278ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:10.782756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1b7e57b9-a78d-4ba2-a5b3-de0fe5280620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '972e72cc6a8441d5beb8ce3a1b6278ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:10.877639 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1b7e57b9-a78d-4ba2-a5b3-de0fe5280620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '972e72cc6a8441d5beb8ce3a1b6278ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:10.878266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1b7e57b9-a78d-4ba2-a5b3-de0fe5280620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '972e72cc6a8441d5beb8ce3a1b6278ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:10.886687 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:16:10.903194 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1107/2184] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:16:10 2022] DELETE /identity/v3/users/972e72cc6a8441d5beb8ce3a1b6278ee => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:16:11.422259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102098ee-71bd-429f-ba94-93b917e9bdd5 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:11.423832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-102098ee-71bd-429f-ba94-93b917e9bdd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:11.440570 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102098ee-71bd-429f-ba94-93b917e9bdd5 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:11.458179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:16:11.497998 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:11.498630 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:11.498907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:11.499210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2 admin admin] PATH_INFO: `/v3/projects/2e0ecfb2fc454932b646f4f15c98c903` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:11.500349 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e0ecfb2fc454932b646f4f15c98c903)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:11.502307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:11.628033 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e0ecfb2fc454932b646f4f15c98c903', 'request_id': 'req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:11.634596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e0ecfb2fc454932b646f4f15c98c903', 'request_id': 'req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:11.746139 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e0ecfb2fc454932b646f4f15c98c903', 'request_id': 'req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:11.757630 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2 admin admin] The token cache is being invalidate because project 2e0ecfb2fc454932b646f4f15c98c903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:16:11.757918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:11.758547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:11.760107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1078/2185] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:16:11 2022] DELETE /identity/v3/projects/2e0ecfb2fc454932b646f4f15c98c903 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:16:11.765599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e57b9-a78d-4ba2-a5b3-de0fe5280620 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:11.765599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1b7e57b9-a78d-4ba2-a5b3-de0fe5280620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:11.778981 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e57b9-a78d-4ba2-a5b3-de0fe5280620 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:11.834017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2be131-405b-4f79-bae9-40e30b9f4bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:11.834649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2be131-405b-4f79-bae9-40e30b9f4bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:11.834926 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2be131-405b-4f79-bae9-40e30b9f4bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:11.835222 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2be131-405b-4f79-bae9-40e30b9f4bc2 admin admin] PATH_INFO: `/v3/users/ec2c3ec9582846bd9f50e1780ab7e377` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:11.836341 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2be131-405b-4f79-bae9-40e30b9f4bc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec2c3ec9582846bd9f50e1780ab7e377)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:11.838827 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2be131-405b-4f79-bae9-40e30b9f4bc2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:12.040609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a2be131-405b-4f79-bae9-40e30b9f4bc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec2c3ec9582846bd9f50e1780ab7e377'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:12.055509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a2be131-405b-4f79-bae9-40e30b9f4bc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec2c3ec9582846bd9f50e1780ab7e377'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:12.056263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a2be131-405b-4f79-bae9-40e30b9f4bc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec2c3ec9582846bd9f50e1780ab7e377'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:12.153006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a2be131-405b-4f79-bae9-40e30b9f4bc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec2c3ec9582846bd9f50e1780ab7e377'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:12.153633 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a2be131-405b-4f79-bae9-40e30b9f4bc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec2c3ec9582846bd9f50e1780ab7e377'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:12.167458 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1108/2186] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:16:11 2022] DELETE /identity/v3/users/ec2c3ec9582846bd9f50e1780ab7e377 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:16:12.627825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:12.630959 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:12.644963 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c69cf-a232-4bb5-af79-ce2b36f5b0b2 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:12.705415 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770bf3f-f48f-435a-b84f-402b56aa8a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:12.706036 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770bf3f-f48f-435a-b84f-402b56aa8a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:12.706317 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770bf3f-f48f-435a-b84f-402b56aa8a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:12.706588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770bf3f-f48f-435a-b84f-402b56aa8a29 admin admin] PATH_INFO: `/v3/projects/74d6395d52bd43498013426ae7e9d00c` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:12.707802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770bf3f-f48f-435a-b84f-402b56aa8a29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74d6395d52bd43498013426ae7e9d00c)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:12.709755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770bf3f-f48f-435a-b84f-402b56aa8a29 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:12.902481 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c770bf3f-f48f-435a-b84f-402b56aa8a29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74d6395d52bd43498013426ae7e9d00c', 'request_id': 'req-c770bf3f-f48f-435a-b84f-402b56aa8a29'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:12.904482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:16:12.908861 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c770bf3f-f48f-435a-b84f-402b56aa8a29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74d6395d52bd43498013426ae7e9d00c', 'request_id': 'req-c770bf3f-f48f-435a-b84f-402b56aa8a29'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:13.014656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c770bf3f-f48f-435a-b84f-402b56aa8a29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74d6395d52bd43498013426ae7e9d00c', 'request_id': 'req-c770bf3f-f48f-435a-b84f-402b56aa8a29'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:13.026588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c770bf3f-f48f-435a-b84f-402b56aa8a29 admin admin] The token cache is being invalidate because project 74d6395d52bd43498013426ae7e9d00c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:16:13.026588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c770bf3f-f48f-435a-b84f-402b56aa8a29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:13.027185 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c770bf3f-f48f-435a-b84f-402b56aa8a29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:16:13.028726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1079/2187] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:16:12 2022] DELETE /identity/v3/projects/74d6395d52bd43498013426ae7e9d00c => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:16:13.033596 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fed8d0-7af2-46a0-9d4b-cd890baea270 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:13.034088 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fed8d0-7af2-46a0-9d4b-cd890baea270 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:13.034193 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fed8d0-7af2-46a0-9d4b-cd890baea270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:13.055060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e4fed8d0-7af2-46a0-9d4b-cd890baea270 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:16:13.056272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e4fed8d0-7af2-46a0-9d4b-cd890baea270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:13.118506 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1109/2188] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:16:13.131432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a18dba-564e-4bbb-be66-1c155fa62821 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:13.131432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a18dba-564e-4bbb-be66-1c155fa62821 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:13.131432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a18dba-564e-4bbb-be66-1c155fa62821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:13.151455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-f3a18dba-564e-4bbb-be66-1c155fa62821 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:16:13.153191 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f3a18dba-564e-4bbb-be66-1c155fa62821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:13.162391 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1080/2189] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:16:13.167848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fed8d0-7af2-46a0-9d4b-cd890baea270 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:13.187807 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fed8d0-7af2-46a0-9d4b-cd890baea270 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:13.187807 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68093ee-6139-430e-8488-ce09c58625ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:13.187807 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68093ee-6139-430e-8488-ce09c58625ba admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:13.187807 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68093ee-6139-430e-8488-ce09c58625ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:13.188269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68093ee-6139-430e-8488-ce09c58625ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:13.188269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68093ee-6139-430e-8488-ce09c58625ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:16:13.188269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68093ee-6139-430e-8488-ce09c58625ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:13.189819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68093ee-6139-430e-8488-ce09c58625ba admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:13.199203 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:16:13.204780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1110/2190] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:16:13.216436 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a18dba-564e-4bbb-be66-1c155fa62821 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:13.227802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a18dba-564e-4bbb-be66-1c155fa62821 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:13.233023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e9ea6-ad95-4459-b555-1245b99228cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:13.240378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e9ea6-ad95-4459-b555-1245b99228cb admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:13.240378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e9ea6-ad95-4459-b555-1245b99228cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:13.240378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e9ea6-ad95-4459-b555-1245b99228cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:13.240378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58e9ea6-ad95-4459-b555-1245b99228cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:13.240378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58e9ea6-ad95-4459-b555-1245b99228cb admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:13.337161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1081/2191] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:16:13 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:16:13.341985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68093ee-6139-430e-8488-ce09c58625ba admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:13.352125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68093ee-6139-430e-8488-ce09c58625ba admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:13.412561 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a38c9-6d3a-4866-926d-b2cdbf32223d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:13.413087 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a38c9-6d3a-4866-926d-b2cdbf32223d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:13.413386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a38c9-6d3a-4866-926d-b2cdbf32223d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:13.413651 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a38c9-6d3a-4866-926d-b2cdbf32223d admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:13.414359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a38c9-6d3a-4866-926d-b2cdbf32223d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:13.416275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a38c9-6d3a-4866-926d-b2cdbf32223d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:13.495239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1111/2192] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:16:13 2022] POST /identity/v3/users => generated 284 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:16:13.499133 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e9ea6-ad95-4459-b555-1245b99228cb admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:13.509771 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e9ea6-ad95-4459-b555-1245b99228cb admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:13.515227 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac141e8-deb0-46b9-b280-b567e76bd3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:13.515816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac141e8-deb0-46b9-b280-b567e76bd3ff admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:13.516091 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac141e8-deb0-46b9-b280-b567e76bd3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:13.516362 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac141e8-deb0-46b9-b280-b567e76bd3ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:13.517001 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac141e8-deb0-46b9-b280-b567e76bd3ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:13.518603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac141e8-deb0-46b9-b280-b567e76bd3ff admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:13.528958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1082/2193] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:16:13.532875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a38c9-6d3a-4866-926d-b2cdbf32223d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:13.548266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a38c9-6d3a-4866-926d-b2cdbf32223d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:13.554498 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197edf78-b40e-4634-b63b-66b9e221bb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:13.555212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197edf78-b40e-4634-b63b-66b9e221bb1e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:13.555489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197edf78-b40e-4634-b63b-66b9e221bb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:13.555840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197edf78-b40e-4634-b63b-66b9e221bb1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:13.556496 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197edf78-b40e-4634-b63b-66b9e221bb1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:13.559666 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197edf78-b40e-4634-b63b-66b9e221bb1e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:13.570795 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1112/2194] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:16:13.575738 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac141e8-deb0-46b9-b280-b567e76bd3ff admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:13.585617 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac141e8-deb0-46b9-b280-b567e76bd3ff admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:13.591298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb6b00-87a9-4eed-a23b-194c64d6c9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:13.593252 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb6b00-87a9-4eed-a23b-194c64d6c9f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:13.593531 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb6b00-87a9-4eed-a23b-194c64d6c9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:13.594226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb6b00-87a9-4eed-a23b-194c64d6c9f8 admin admin] PATH_INFO: `/v3/projects/37fabc1e83cf476286a4b7bdcbd91576/users/1e3e8d6251204e8385d17a9a27be091c/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:13.611148 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb6b00-87a9-4eed-a23b-194c64d6c9f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37fabc1e83cf476286a4b7bdcbd91576, user_id=1e3e8d6251204e8385d17a9a27be091c, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:13.614439 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb6b00-87a9-4eed-a23b-194c64d6c9f8 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:13.719671 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1083/2195] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:16:13 2022] PUT /identity/v3/projects/37fabc1e83cf476286a4b7bdcbd91576/users/1e3e8d6251204e8385d17a9a27be091c/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:16:13.730552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d18c59-4392-44ff-88dd-8f8757e72d7e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:13.730822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d18c59-4392-44ff-88dd-8f8757e72d7e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:13.731121 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d18c59-4392-44ff-88dd-8f8757e72d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:13.759988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c0d18c59-4392-44ff-88dd-8f8757e72d7e None None] MFA Rules not processed for user `1e3e8d6251204e8385d17a9a27be091c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:16:13.761292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c0d18c59-4392-44ff-88dd-8f8757e72d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:13.842435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1113/2196] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:16:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:16:13.855183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb6b00-87a9-4eed-a23b-194c64d6c9f8 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:13.857169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9acb6b00-87a9-4eed-a23b-194c64d6c9f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:13.944652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb6b00-87a9-4eed-a23b-194c64d6c9f8 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:13.950416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5871e-1919-4424-a9f7-5f80480216c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30909d152a5c4a1fac6c1a2765868177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:13.951488 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5871e-1919-4424-a9f7-5f80480216c7 service glance] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:13.952010 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5871e-1919-4424-a9f7-5f80480216c7 service glance] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:13.952408 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5871e-1919-4424-a9f7-5f80480216c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:13.962709 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5871e-1919-4424-a9f7-5f80480216c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:13.965016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5871e-1919-4424-a9f7-5f80480216c7 service glance] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:13.975535 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1084/2197] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:16:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:16:23.727920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dbc58-7181-4b06-a40e-ee5b422eb3e0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:23.734302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dbc58-7181-4b06-a40e-ee5b422eb3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:23.734302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dbc58-7181-4b06-a40e-ee5b422eb3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:23.751528 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ff1dbc58-7181-4b06-a40e-ee5b422eb3e0 None None] MFA Rules not processed for user `85bdfc4b6ad443d68ebc0d6932ccad66`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:16:23.752849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff1dbc58-7181-4b06-a40e-ee5b422eb3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:23.819824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1114/2198] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:16:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:16:23.970562 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=58,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:16:33.812755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:16:34.933663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5871e-1919-4424-a9f7-5f80480216c7 service glance] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:34.935567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b9f5871e-1919-4424-a9f7-5f80480216c7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:34.994643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5871e-1919-4424-a9f7-5f80480216c7 service glance] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:34.999858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71acdb05-2dcd-4c93-a906-ffe52ca5c89d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:35.000475 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acdb05-2dcd-4c93-a906-ffe52ca5c89d service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:35.000750 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acdb05-2dcd-4c93-a906-ffe52ca5c89d service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:35.001020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acdb05-2dcd-4c93-a906-ffe52ca5c89d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:35.002778 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-71acdb05-2dcd-4c93-a906-ffe52ca5c89d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:35.068871 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71acdb05-2dcd-4c93-a906-ffe52ca5c89d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:35.070318 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71acdb05-2dcd-4c93-a906-ffe52ca5c89d service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:35.080462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1085/2199] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:16:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:16:38.008516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1dbc58-7181-4b06-a40e-ee5b422eb3e0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:16:38.009963 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff1dbc58-7181-4b06-a40e-ee5b422eb3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:38.164008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1dbc58-7181-4b06-a40e-ee5b422eb3e0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:16:38.174460 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7a916-bbb6-4e8b-a10f-d6baa0ee1307 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6074872303d3468fb566378fa34e8bea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:16:38.175481 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7a916-bbb6-4e8b-a10f-d6baa0ee1307 service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:16:38.176230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7a916-bbb6-4e8b-a10f-d6baa0ee1307 service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:16:38.176502 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7a916-bbb6-4e8b-a10f-d6baa0ee1307 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:16:38.178440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cbc7a916-bbb6-4e8b-a10f-d6baa0ee1307 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:16:38.208335 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7a916-bbb6-4e8b-a10f-d6baa0ee1307 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:16:38.209846 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7a916-bbb6-4e8b-a10f-d6baa0ee1307 service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:16:38.222988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1115/2200] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:16:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:16:45.076046 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=50 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:16:48.217130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=32 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:17:10.867291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a7cc0-8e56-4cf9-b96b-f90dfe09d6a7 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:10.867824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a7cc0-8e56-4cf9-b96b-f90dfe09d6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:10.867824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a7cc0-8e56-4cf9-b96b-f90dfe09d6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:10.888350 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-f82a7cc0-8e56-4cf9-b96b-f90dfe09d6a7 None None] MFA Rules not processed for user `9932c7fb57394d709a27023f3ae2677e`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:17:10.889558 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f82a7cc0-8e56-4cf9-b96b-f90dfe09d6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:10.944038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1086/2201] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:17:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:17:14.069404 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7a916-bbb6-4e8b-a10f-d6baa0ee1307 service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:17:14.142492 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7a916-bbb6-4e8b-a10f-d6baa0ee1307 service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:17:14.147521 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab4a63-554e-4c2a-a5cd-89639c24cb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:17:14.148292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab4a63-554e-4c2a-a5cd-89639c24cb0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:14.148492 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab4a63-554e-4c2a-a5cd-89639c24cb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:14.148767 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab4a63-554e-4c2a-a5cd-89639c24cb0f admin admin] PATH_INFO: `/v3/users/1e3e8d6251204e8385d17a9a27be091c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:14.149990 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab4a63-554e-4c2a-a5cd-89639c24cb0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e3e8d6251204e8385d17a9a27be091c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:17:14.151965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab4a63-554e-4c2a-a5cd-89639c24cb0f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:17:14.333851 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-efab4a63-554e-4c2a-a5cd-89639c24cb0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e3e8d6251204e8385d17a9a27be091c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:17:14.345802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-efab4a63-554e-4c2a-a5cd-89639c24cb0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e3e8d6251204e8385d17a9a27be091c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:17:14.346474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-efab4a63-554e-4c2a-a5cd-89639c24cb0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e3e8d6251204e8385d17a9a27be091c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:17:14.431995 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-efab4a63-554e-4c2a-a5cd-89639c24cb0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e3e8d6251204e8385d17a9a27be091c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:17:14.439352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-efab4a63-554e-4c2a-a5cd-89639c24cb0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e3e8d6251204e8385d17a9a27be091c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:17:14.447674 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1116/2202] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:17:14 2022] DELETE /identity/v3/users/1e3e8d6251204e8385d17a9a27be091c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:17:14.452262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07b732-a2f9-44bc-a477-5a903d2cd83a None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:14.452736 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07b732-a2f9-44bc-a477-5a903d2cd83a None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:14.452848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07b732-a2f9-44bc-a477-5a903d2cd83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:14.475392 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-3e07b732-a2f9-44bc-a477-5a903d2cd83a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:17:14.476615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3e07b732-a2f9-44bc-a477-5a903d2cd83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:14.540544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1087/2203] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:17:14.550586 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab4a63-554e-4c2a-a5cd-89639c24cb0f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:17:14.552284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-efab4a63-554e-4c2a-a5cd-89639c24cb0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:14.627501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab4a63-554e-4c2a-a5cd-89639c24cb0f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:17:14.632090 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56f438-f534-4285-931b-06369e70caec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:17:14.632858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56f438-f534-4285-931b-06369e70caec service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:14.633146 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56f438-f534-4285-931b-06369e70caec service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:14.633421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56f438-f534-4285-931b-06369e70caec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:14.643172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56f438-f534-4285-931b-06369e70caec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:17:14.645374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56f438-f534-4285-931b-06369e70caec service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:17:14.655473 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1117/2204] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:17:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:17:15.078702 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07b732-a2f9-44bc-a477-5a903d2cd83a None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:17:15.080153 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3e07b732-a2f9-44bc-a477-5a903d2cd83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:15.093288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07b732-a2f9-44bc-a477-5a903d2cd83a None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:17:15.097987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430d9bb-83b1-455c-a1a6-3455f4dc6e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:17:15.098615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430d9bb-83b1-455c-a1a6-3455f4dc6e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:15.098970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430d9bb-83b1-455c-a1a6-3455f4dc6e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:15.099238 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430d9bb-83b1-455c-a1a6-3455f4dc6e5c admin admin] PATH_INFO: `/v3/projects/37fabc1e83cf476286a4b7bdcbd91576` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:15.100418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430d9bb-83b1-455c-a1a6-3455f4dc6e5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=37fabc1e83cf476286a4b7bdcbd91576)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:17:15.102324 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430d9bb-83b1-455c-a1a6-3455f4dc6e5c admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:17:15.217086 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-1430d9bb-83b1-455c-a1a6-3455f4dc6e5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37fabc1e83cf476286a4b7bdcbd91576', 'request_id': 'req-1430d9bb-83b1-455c-a1a6-3455f4dc6e5c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:17:15.223928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-1430d9bb-83b1-455c-a1a6-3455f4dc6e5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37fabc1e83cf476286a4b7bdcbd91576', 'request_id': 'req-1430d9bb-83b1-455c-a1a6-3455f4dc6e5c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:17:15.305568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-1430d9bb-83b1-455c-a1a6-3455f4dc6e5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37fabc1e83cf476286a4b7bdcbd91576', 'request_id': 'req-1430d9bb-83b1-455c-a1a6-3455f4dc6e5c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:17:15.316742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-1430d9bb-83b1-455c-a1a6-3455f4dc6e5c admin admin] The token cache is being invalidate because project 37fabc1e83cf476286a4b7bdcbd91576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:17:15.317040 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-1430d9bb-83b1-455c-a1a6-3455f4dc6e5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:17:15.317659 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-1430d9bb-83b1-455c-a1a6-3455f4dc6e5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:17:15.319192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1088/2205] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:17:15 2022] DELETE /identity/v3/projects/37fabc1e83cf476286a4b7bdcbd91576 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:17:15.324177 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ad32a-1653-4907-a026-87918e60454b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:15.324628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ad32a-1653-4907-a026-87918e60454b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:15.324724 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ad32a-1653-4907-a026-87918e60454b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:15.344741 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-287ad32a-1653-4907-a026-87918e60454b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:17:15.345932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-287ad32a-1653-4907-a026-87918e60454b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:15.363728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:17:15.405717 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1118/2206] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:17:15.413542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b748c8c-b1e1-476c-9448-df846ad435d4 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:15.413832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b748c8c-b1e1-476c-9448-df846ad435d4 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:15.414111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b748c8c-b1e1-476c-9448-df846ad435d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:15.439233 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-6b748c8c-b1e1-476c-9448-df846ad435d4 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:17:15.441989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6b748c8c-b1e1-476c-9448-df846ad435d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:15.453058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1089/2207] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:17:15.457268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ad32a-1653-4907-a026-87918e60454b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:17:15.475532 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ad32a-1653-4907-a026-87918e60454b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:17:15.481474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e4d78-278a-4f2b-bc04-24e0494f11bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:17:15.482650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e4d78-278a-4f2b-bc04-24e0494f11bf admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:15.482650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e4d78-278a-4f2b-bc04-24e0494f11bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:15.484227 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e4d78-278a-4f2b-bc04-24e0494f11bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:15.485063 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e4d78-278a-4f2b-bc04-24e0494f11bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:17:15.485558 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e4d78-278a-4f2b-bc04-24e0494f11bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:17:15.487914 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e4d78-278a-4f2b-bc04-24e0494f11bf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:17:15.502405 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1119/2208] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:17:15.521325 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b748c8c-b1e1-476c-9448-df846ad435d4 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:17:15.521325 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b748c8c-b1e1-476c-9448-df846ad435d4 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:17:15.526059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014da063-68f5-42ba-b146-8ada25061e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:17:15.526907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014da063-68f5-42ba-b146-8ada25061e01 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:15.527300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014da063-68f5-42ba-b146-8ada25061e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:15.527645 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014da063-68f5-42ba-b146-8ada25061e01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:15.528355 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014da063-68f5-42ba-b146-8ada25061e01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:17:15.530567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014da063-68f5-42ba-b146-8ada25061e01 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:17:15.633342 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:17:15.643255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1090/2209] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:17:15 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:17:15.644715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e4d78-278a-4f2b-bc04-24e0494f11bf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:17:15.654708 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e4d78-278a-4f2b-bc04-24e0494f11bf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:17:15.720982 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbae2a7-80d5-438a-88fb-980a2cfd1bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:17:15.722180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbae2a7-80d5-438a-88fb-980a2cfd1bbd admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:15.722535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbae2a7-80d5-438a-88fb-980a2cfd1bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:15.722808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbae2a7-80d5-438a-88fb-980a2cfd1bbd admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:15.723483 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbae2a7-80d5-438a-88fb-980a2cfd1bbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:17:15.725431 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbae2a7-80d5-438a-88fb-980a2cfd1bbd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:17:15.805410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1120/2210] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:17:15 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:17:15.809656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014da063-68f5-42ba-b146-8ada25061e01 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:17:15.825585 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014da063-68f5-42ba-b146-8ada25061e01 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:17:15.832176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9987fc0-87eb-4976-9456-4bb71b7ed0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:17:15.833697 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9987fc0-87eb-4976-9456-4bb71b7ed0ed admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:15.833975 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9987fc0-87eb-4976-9456-4bb71b7ed0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:15.834249 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9987fc0-87eb-4976-9456-4bb71b7ed0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:15.834913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9987fc0-87eb-4976-9456-4bb71b7ed0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:17:15.836850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9987fc0-87eb-4976-9456-4bb71b7ed0ed admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:17:15.852802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1091/2211] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:17:15.854013 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbae2a7-80d5-438a-88fb-980a2cfd1bbd admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:17:15.868635 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbae2a7-80d5-438a-88fb-980a2cfd1bbd admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:17:15.884052 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292d287-ff23-4914-bbdc-ae1e953beb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:17:15.885371 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292d287-ff23-4914-bbdc-ae1e953beb6d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:15.885371 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292d287-ff23-4914-bbdc-ae1e953beb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:15.885371 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292d287-ff23-4914-bbdc-ae1e953beb6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:15.886326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292d287-ff23-4914-bbdc-ae1e953beb6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:17:15.888515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292d287-ff23-4914-bbdc-ae1e953beb6d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:17:15.901472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1121/2212] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:17:15.905743 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9987fc0-87eb-4976-9456-4bb71b7ed0ed admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:17:15.916419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9987fc0-87eb-4976-9456-4bb71b7ed0ed admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:17:15.921150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d44a9-b0df-4fdb-8bc7-605639c788fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:17:15.921786 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d44a9-b0df-4fdb-8bc7-605639c788fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:15.922060 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d44a9-b0df-4fdb-8bc7-605639c788fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:15.922331 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d44a9-b0df-4fdb-8bc7-605639c788fd admin admin] PATH_INFO: `/v3/projects/35d7501875734ba1a41351b2ac57db09/users/2d2a46971e1e459e92fabdc8a5c54d10/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:15.938813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d44a9-b0df-4fdb-8bc7-605639c788fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=35d7501875734ba1a41351b2ac57db09, user_id=2d2a46971e1e459e92fabdc8a5c54d10, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:17:15.941769 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d44a9-b0df-4fdb-8bc7-605639c788fd admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:17:16.016534 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1092/2213] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:17:15 2022] PUT /identity/v3/projects/35d7501875734ba1a41351b2ac57db09/users/2d2a46971e1e459e92fabdc8a5c54d10/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:17:16.028328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5860c0b-7cc6-4164-bd4b-c8b51a578841 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:16.028328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5860c0b-7cc6-4164-bd4b-c8b51a578841 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:16.028328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5860c0b-7cc6-4164-bd4b-c8b51a578841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:16.078327 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c5860c0b-7cc6-4164-bd4b-c8b51a578841 None None] MFA Rules not processed for user `2d2a46971e1e459e92fabdc8a5c54d10`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:17:16.079531 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c5860c0b-7cc6-4164-bd4b-c8b51a578841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:16.190573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1122/2214] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:17:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:17:16.201756 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d44a9-b0df-4fdb-8bc7-605639c788fd admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:17:16.203557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1b6d44a9-b0df-4fdb-8bc7-605639c788fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:16.263494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d44a9-b0df-4fdb-8bc7-605639c788fd admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:17:16.267327 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968b3c8-1c98-43bb-906d-e694e6031397 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:17:16.267929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968b3c8-1c98-43bb-906d-e694e6031397 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:16.268210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968b3c8-1c98-43bb-906d-e694e6031397 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:16.269011 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968b3c8-1c98-43bb-906d-e694e6031397 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:16.282495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968b3c8-1c98-43bb-906d-e694e6031397 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:17:16.284272 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968b3c8-1c98-43bb-906d-e694e6031397 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:17:16.295999 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1093/2215] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:17:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:17:19.127879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a7c8f-c309-4898-ab1d-5947cf9f8c07 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:19.128983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a7c8f-c309-4898-ab1d-5947cf9f8c07 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:19.129080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a7c8f-c309-4898-ab1d-5947cf9f8c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:19.149996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1c9a7c8f-c309-4898-ab1d-5947cf9f8c07 None None] MFA Rules not processed for user `2d2a46971e1e459e92fabdc8a5c54d10`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:17:19.151239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1c9a7c8f-c309-4898-ab1d-5947cf9f8c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:19.158311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1123/2216] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:17:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:17:26.291306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:17:29.144564 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:17:31.568707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33a31b-04b1-4197-b5b2-eba75a6415b5 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:31.569211 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33a31b-04b1-4197-b5b2-eba75a6415b5 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:31.569320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33a31b-04b1-4197-b5b2-eba75a6415b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:31.593381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-0c33a31b-04b1-4197-b5b2-eba75a6415b5 None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:17:31.594659 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0c33a31b-04b1-4197-b5b2-eba75a6415b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:31.668372 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1094/2217] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Oct 6 08:17:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:17:31.715633 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a7c8f-c309-4898-ab1d-5947cf9f8c07 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:17:31.717272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1c9a7c8f-c309-4898-ab1d-5947cf9f8c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:31.803001 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a7c8f-c309-4898-ab1d-5947cf9f8c07 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:17:31.809294 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97a251-5fce-4103-bc88-f09efa1824a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:17:31.809850 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97a251-5fce-4103-bc88-f09efa1824a0 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:31.810129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97a251-5fce-4103-bc88-f09efa1824a0 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:31.812899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97a251-5fce-4103-bc88-f09efa1824a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:31.820813 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97a251-5fce-4103-bc88-f09efa1824a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:17:31.822454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97a251-5fce-4103-bc88-f09efa1824a0 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:17:31.834703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1124/2218] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:17:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:17:41.667189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=25 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:17:41.828287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=32 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:17:54.669292 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad37fa-4a10-400c-8749-854ea18ea4a4 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:54.669883 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad37fa-4a10-400c-8749-854ea18ea4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:54.669883 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad37fa-4a10-400c-8749-854ea18ea4a4 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:54.671271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1095/2219] 10.222.0.30 () {54 vars in 921 bytes} [Thu Oct 6 08:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:17:54.677169 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3467bb-0783-4f7a-9a2e-ed7a2ef7539a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:54.678288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3467bb-0783-4f7a-9a2e-ed7a2ef7539a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:54.678749 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3467bb-0783-4f7a-9a2e-ed7a2ef7539a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:54.698457 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3a3467bb-0783-4f7a-9a2e-ed7a2ef7539a None None] MFA Rules not processed for user `209795285384486183cda455fe316bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:17:54.699703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3a3467bb-0783-4f7a-9a2e-ed7a2ef7539a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:54.766924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1125/2220] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Oct 6 08:17:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:17:54.778408 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad37fa-4a10-400c-8749-854ea18ea4a4 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:17:54.780151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-27ad37fa-4a10-400c-8749-854ea18ea4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:54.838517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad37fa-4a10-400c-8749-854ea18ea4a4 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:17:54.842077 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccdab31-e95f-488c-9fe4-b95fdf8fd5d1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30909d152a5c4a1fac6c1a2765868177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:17:54.842668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdab31-e95f-488c-9fe4-b95fdf8fd5d1 service glance] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:54.842972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdab31-e95f-488c-9fe4-b95fdf8fd5d1 service glance] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:54.843236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdab31-e95f-488c-9fe4-b95fdf8fd5d1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:54.853645 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccdab31-e95f-488c-9fe4-b95fdf8fd5d1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:17:54.855095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccdab31-e95f-488c-9fe4-b95fdf8fd5d1 service glance] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:17:54.866241 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1096/2221] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:17:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:17:55.022162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8982f9-f826-4824-b5ef-cbbadb7dc723 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:55.022440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8982f9-f826-4824-b5ef-cbbadb7dc723 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:55.022705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8982f9-f826-4824-b5ef-cbbadb7dc723 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:55.024108 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1126/2222] 10.222.0.30 () {54 vars in 921 bytes} [Thu Oct 6 08:17:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:17:55.028768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3eaa6-2fec-4855-87fb-cd1bd2298e45 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:55.029044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3eaa6-2fec-4855-87fb-cd1bd2298e45 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:55.029310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3eaa6-2fec-4855-87fb-cd1bd2298e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:55.050571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-f5b3eaa6-2fec-4855-87fb-cd1bd2298e45 None None] MFA Rules not processed for user `bf6798f89acf44e897c481924edd9047`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:17:55.051786 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f5b3eaa6-2fec-4855-87fb-cd1bd2298e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:55.102101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1097/2223] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Oct 6 08:17:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:17:55.114960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8982f9-f826-4824-b5ef-cbbadb7dc723 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:17:55.117160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4e8982f9-f826-4824-b5ef-cbbadb7dc723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:55.133098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8982f9-f826-4824-b5ef-cbbadb7dc723 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:17:55.135688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f4f91-958f-44ae-808d-8d42d55236ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:17:55.136362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f4f91-958f-44ae-808d-8d42d55236ce service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:55.136641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f4f91-958f-44ae-808d-8d42d55236ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:55.136906 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f4f91-958f-44ae-808d-8d42d55236ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:55.149990 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f4f91-958f-44ae-808d-8d42d55236ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:17:55.151635 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f4f91-958f-44ae-808d-8d42d55236ce service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:17:55.159937 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1127/2224] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:17:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:17:55.264256 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe000955-cb78-4260-8e9b-a1a83a5280b2 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:55.264736 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe000955-cb78-4260-8e9b-a1a83a5280b2 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:55.264997 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe000955-cb78-4260-8e9b-a1a83a5280b2 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:55.266764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1098/2225] 10.222.0.30 () {54 vars in 921 bytes} [Thu Oct 6 08:17:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:17:55.273241 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8e791-361f-4b89-a450-3da1f47f23f9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:55.273501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8e791-361f-4b89-a450-3da1f47f23f9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:55.273749 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8e791-361f-4b89-a450-3da1f47f23f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:55.295842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-54d8e791-361f-4b89-a450-3da1f47f23f9 None None] MFA Rules not processed for user `209795285384486183cda455fe316bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:17:55.297648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-54d8e791-361f-4b89-a450-3da1f47f23f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:55.309065 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1128/2226] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Oct 6 08:17:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:17:55.320556 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe000955-cb78-4260-8e9b-a1a83a5280b2 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:17:55.331149 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-fe000955-cb78-4260-8e9b-a1a83a5280b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:17:55.336708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe000955-cb78-4260-8e9b-a1a83a5280b2 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:17:55.340565 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42271ee-3af9-4b11-8957-4fe687986b7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:17:55.341163 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42271ee-3af9-4b11-8957-4fe687986b7b service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:17:55.341433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42271ee-3af9-4b11-8957-4fe687986b7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:17:55.341699 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42271ee-3af9-4b11-8957-4fe687986b7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:17:55.354567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42271ee-3af9-4b11-8957-4fe687986b7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:17:55.359715 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42271ee-3af9-4b11-8957-4fe687986b7b service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:17:55.368471 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1099/2227] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:17:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:18:04.066236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5cc93-d24d-4924-a511-364337c23cf7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:04.066712 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5cc93-d24d-4924-a511-364337c23cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:04.066908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5cc93-d24d-4924-a511-364337c23cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:04.086887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-93e5cc93-d24d-4924-a511-364337c23cf7 None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:18:04.088274 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-93e5cc93-d24d-4924-a511-364337c23cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:04.095609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1129/2228] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Oct 6 08:18:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:18:04.108138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42271ee-3af9-4b11-8957-4fe687986b7b service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:18:04.120185 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42271ee-3af9-4b11-8957-4fe687986b7b service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:18:04.125806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af0d23-edb4-4efe-8a65-40f01e796bbc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:18:04.126415 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af0d23-edb4-4efe-8a65-40f01e796bbc service swift] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:04.126691 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af0d23-edb4-4efe-8a65-40f01e796bbc service swift] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:04.127068 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af0d23-edb4-4efe-8a65-40f01e796bbc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:04.136869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af0d23-edb4-4efe-8a65-40f01e796bbc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:18:04.141932 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af0d23-edb4-4efe-8a65-40f01e796bbc service swift] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:18:04.147627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1100/2229] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:18:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:18:05.170776 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549fb07-e2e0-497f-8347-23ec69b0407e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:05.171278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549fb07-e2e0-497f-8347-23ec69b0407e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:05.171372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549fb07-e2e0-497f-8347-23ec69b0407e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:05.190744 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c549fb07-e2e0-497f-8347-23ec69b0407e None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:18:05.191972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c549fb07-e2e0-497f-8347-23ec69b0407e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:05.198919 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1130/2230] 10.222.0.30 () {58 vars in 967 bytes} [Thu Oct 6 08:18:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:18:05.211205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af0d23-edb4-4efe-8a65-40f01e796bbc service swift] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:18:05.221640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af0d23-edb4-4efe-8a65-40f01e796bbc service swift] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:18:05.228095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df98a1c-b8b2-4b11-879a-98379934a237 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:18:05.228688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df98a1c-b8b2-4b11-879a-98379934a237 service swift] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:05.239338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df98a1c-b8b2-4b11-879a-98379934a237 service swift] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:05.239338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df98a1c-b8b2-4b11-879a-98379934a237 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:05.239338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df98a1c-b8b2-4b11-879a-98379934a237 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:18:05.241234 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df98a1c-b8b2-4b11-879a-98379934a237 service swift] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:18:05.249543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1101/2231] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:18:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:18:05.921536 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fe317-0693-4e1e-ac6e-eb0d5f210399 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:05.921830 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fe317-0693-4e1e-ac6e-eb0d5f210399 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:05.922107 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fe317-0693-4e1e-ac6e-eb0d5f210399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:05.941147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b63fe317-0693-4e1e-ac6e-eb0d5f210399 None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:18:05.942321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b63fe317-0693-4e1e-ac6e-eb0d5f210399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:05.950066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1131/2232] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Oct 6 08:18:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:18:05.962896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df98a1c-b8b2-4b11-879a-98379934a237 service swift] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:18:05.973580 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df98a1c-b8b2-4b11-879a-98379934a237 service swift] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:18:05.978837 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccc544-1f69-488e-a503-b94cd350a5f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:18:05.979472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccc544-1f69-488e-a503-b94cd350a5f9 service swift] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:05.979776 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccc544-1f69-488e-a503-b94cd350a5f9 service swift] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:05.980042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccc544-1f69-488e-a503-b94cd350a5f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:05.988507 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccc544-1f69-488e-a503-b94cd350a5f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:18:05.990084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccc544-1f69-488e-a503-b94cd350a5f9 service swift] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:18:05.998121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1102/2233] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:18:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:18:06.601332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd6bef-4fbc-487e-9d5d-e96e61ea4655 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:06.601805 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd6bef-4fbc-487e-9d5d-e96e61ea4655 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:06.601892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd6bef-4fbc-487e-9d5d-e96e61ea4655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:06.622727 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f6fd6bef-4fbc-487e-9d5d-e96e61ea4655 None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:18:06.624026 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f6fd6bef-4fbc-487e-9d5d-e96e61ea4655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:06.631120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1132/2234] 10.222.0.30 () {58 vars in 967 bytes} [Thu Oct 6 08:18:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:18:06.643519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccc544-1f69-488e-a503-b94cd350a5f9 service swift] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:18:06.653832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccc544-1f69-488e-a503-b94cd350a5f9 service swift] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:18:06.658894 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b27bb1-6ce3-41fe-916a-d26ec78388fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:18:06.659510 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b27bb1-6ce3-41fe-916a-d26ec78388fe service swift] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:06.659782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b27bb1-6ce3-41fe-916a-d26ec78388fe service swift] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:06.660048 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b27bb1-6ce3-41fe-916a-d26ec78388fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:06.668544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b27bb1-6ce3-41fe-916a-d26ec78388fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:18:06.670139 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b27bb1-6ce3-41fe-916a-d26ec78388fe service swift] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:18:06.678997 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1103/2235] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:18:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:18:08.293001 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a24f8-603d-4ea2-b924-12cb0c012793 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:08.293556 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a24f8-603d-4ea2-b924-12cb0c012793 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:08.293731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a24f8-603d-4ea2-b924-12cb0c012793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:08.315471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-745a24f8-603d-4ea2-b924-12cb0c012793 None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:18:08.316736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-745a24f8-603d-4ea2-b924-12cb0c012793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:08.324122 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1133/2236] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Oct 6 08:18:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:18:08.341629 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b27bb1-6ce3-41fe-916a-d26ec78388fe service swift] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:18:08.349139 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b27bb1-6ce3-41fe-916a-d26ec78388fe service swift] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:18:08.354967 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c3815-5965-4cbd-b458-cbfd654a33ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:18:08.355609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c3815-5965-4cbd-b458-cbfd654a33ee service swift] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:08.355887 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c3815-5965-4cbd-b458-cbfd654a33ee service swift] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:08.356157 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c3815-5965-4cbd-b458-cbfd654a33ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:08.364351 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c3815-5965-4cbd-b458-cbfd654a33ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:18:08.366608 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c3815-5965-4cbd-b458-cbfd654a33ee service swift] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:18:08.375258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1104/2237] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:18:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:18:09.463885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a24f8-603d-4ea2-b924-12cb0c012793 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:18:09.534597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a24f8-603d-4ea2-b924-12cb0c012793 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:18:09.539689 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f45fb-f937-4e7f-a222-8e5f47cab4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:18:09.540322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f45fb-f937-4e7f-a222-8e5f47cab4c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:09.540604 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f45fb-f937-4e7f-a222-8e5f47cab4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:09.540877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f45fb-f937-4e7f-a222-8e5f47cab4c2 admin admin] PATH_INFO: `/v3/users/2d2a46971e1e459e92fabdc8a5c54d10` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:09.542011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f45fb-f937-4e7f-a222-8e5f47cab4c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d2a46971e1e459e92fabdc8a5c54d10)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:18:09.543995 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f45fb-f937-4e7f-a222-8e5f47cab4c2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:18:09.751880 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:18:09.755720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b97f45fb-f937-4e7f-a222-8e5f47cab4c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d2a46971e1e459e92fabdc8a5c54d10'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:18:09.771486 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b97f45fb-f937-4e7f-a222-8e5f47cab4c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d2a46971e1e459e92fabdc8a5c54d10'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:18:09.772917 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b97f45fb-f937-4e7f-a222-8e5f47cab4c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d2a46971e1e459e92fabdc8a5c54d10'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:18:09.854558 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b97f45fb-f937-4e7f-a222-8e5f47cab4c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d2a46971e1e459e92fabdc8a5c54d10'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:18:09.855336 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b97f45fb-f937-4e7f-a222-8e5f47cab4c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d2a46971e1e459e92fabdc8a5c54d10'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:18:09.868968 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1134/2238] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:18:09 2022] DELETE /identity/v3/users/2d2a46971e1e459e92fabdc8a5c54d10 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:18:09.873460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919eaa1-bab0-49d1-ba37-6b6db20656fd None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:09.873954 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919eaa1-bab0-49d1-ba37-6b6db20656fd None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:09.874061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919eaa1-bab0-49d1-ba37-6b6db20656fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:09.896103 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-1919eaa1-bab0-49d1-ba37-6b6db20656fd None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:18:09.897452 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1919eaa1-bab0-49d1-ba37-6b6db20656fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:09.931123 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:18:09.964310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1105/2239] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:18:09.975693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f45fb-f937-4e7f-a222-8e5f47cab4c2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:18:09.977330 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b97f45fb-f937-4e7f-a222-8e5f47cab4c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:10.046414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f45fb-f937-4e7f-a222-8e5f47cab4c2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:18:10.050146 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5eb74-0865-4747-a108-3924fb54a7be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:18:10.050751 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5eb74-0865-4747-a108-3924fb54a7be service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:10.051067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5eb74-0865-4747-a108-3924fb54a7be service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:10.051339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5eb74-0865-4747-a108-3924fb54a7be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:10.060960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5eb74-0865-4747-a108-3924fb54a7be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:18:10.062381 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5eb74-0865-4747-a108-3924fb54a7be service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:18:10.074504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1135/2240] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:18:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:18:10.627484 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919eaa1-bab0-49d1-ba37-6b6db20656fd None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:18:10.628991 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1919eaa1-bab0-49d1-ba37-6b6db20656fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:10.644948 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919eaa1-bab0-49d1-ba37-6b6db20656fd None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:18:10.650823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63752b54-806e-4529-a150-daa5766575d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:18:10.651795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63752b54-806e-4529-a150-daa5766575d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:10.652519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63752b54-806e-4529-a150-daa5766575d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:10.653670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63752b54-806e-4529-a150-daa5766575d7 admin admin] PATH_INFO: `/v3/projects/35d7501875734ba1a41351b2ac57db09` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:10.655181 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63752b54-806e-4529-a150-daa5766575d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35d7501875734ba1a41351b2ac57db09)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:18:10.658201 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63752b54-806e-4529-a150-daa5766575d7 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:18:10.769068 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-63752b54-806e-4529-a150-daa5766575d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35d7501875734ba1a41351b2ac57db09', 'request_id': 'req-63752b54-806e-4529-a150-daa5766575d7'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:18:10.776644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-63752b54-806e-4529-a150-daa5766575d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35d7501875734ba1a41351b2ac57db09', 'request_id': 'req-63752b54-806e-4529-a150-daa5766575d7'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:18:10.843284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-63752b54-806e-4529-a150-daa5766575d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35d7501875734ba1a41351b2ac57db09', 'request_id': 'req-63752b54-806e-4529-a150-daa5766575d7'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:18:10.856640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-63752b54-806e-4529-a150-daa5766575d7 admin admin] The token cache is being invalidate because project 35d7501875734ba1a41351b2ac57db09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:18:10.857257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-63752b54-806e-4529-a150-daa5766575d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:18:10.858044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-63752b54-806e-4529-a150-daa5766575d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:18:10.859732 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1106/2241] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:18:10 2022] DELETE /identity/v3/projects/35d7501875734ba1a41351b2ac57db09 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:18:18.028947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc70e6-5d05-43fe-81d4-d66dbba93dde None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:18.029671 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc70e6-5d05-43fe-81d4-d66dbba93dde None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:18.029671 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc70e6-5d05-43fe-81d4-d66dbba93dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:18.053996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f9cc70e6-5d05-43fe-81d4-d66dbba93dde None None] MFA Rules not processed for user `fff5a88cba04475c9a0142156a2269f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:18:18.055239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f9cc70e6-5d05-43fe-81d4-d66dbba93dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:18.112622 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1136/2242] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:18:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:18:20.852013 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:18:23.265705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63752b54-806e-4529-a150-daa5766575d7 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:18:23.267134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-63752b54-806e-4529-a150-daa5766575d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:23.323283 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63752b54-806e-4529-a150-daa5766575d7 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:18:23.326973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20983d-affc-42d0-9319-2134a775fbe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:18:23.327559 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20983d-affc-42d0-9319-2134a775fbe2 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:23.327830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20983d-affc-42d0-9319-2134a775fbe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:23.328093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20983d-affc-42d0-9319-2134a775fbe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:23.329773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-fe20983d-affc-42d0-9319-2134a775fbe2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:23.411288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20983d-affc-42d0-9319-2134a775fbe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:18:23.412729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20983d-affc-42d0-9319-2134a775fbe2 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:18:23.423814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1107/2243] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:18:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:18:26.007627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc70e6-5d05-43fe-81d4-d66dbba93dde None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:18:26.009362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f9cc70e6-5d05-43fe-81d4-d66dbba93dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:26.020831 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc70e6-5d05-43fe-81d4-d66dbba93dde None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:18:26.024331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1758a9-36dd-4940-81d4-e3a71d3c4465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:18:26.024915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1758a9-36dd-4940-81d4-e3a71d3c4465 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:26.025184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1758a9-36dd-4940-81d4-e3a71d3c4465 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:26.025446 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1758a9-36dd-4940-81d4-e3a71d3c4465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:26.027085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4c1758a9-36dd-4940-81d4-e3a71d3c4465 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:26.040248 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1758a9-36dd-4940-81d4-e3a71d3c4465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:18:26.041651 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1758a9-36dd-4940-81d4-e3a71d3c4465 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:18:26.053395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1137/2244] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:18:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:18:33.418751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=54 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:18:36.048800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=71,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:18:39.065477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20983d-affc-42d0-9319-2134a775fbe2 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:18:39.066970 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-fe20983d-affc-42d0-9319-2134a775fbe2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:39.080995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20983d-affc-42d0-9319-2134a775fbe2 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:18:39.086478 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702ed88-0783-4dbf-b8d4-3db46495389c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:18:39.087092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702ed88-0783-4dbf-b8d4-3db46495389c service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:39.087369 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702ed88-0783-4dbf-b8d4-3db46495389c service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:39.087637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702ed88-0783-4dbf-b8d4-3db46495389c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:39.089309 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f702ed88-0783-4dbf-b8d4-3db46495389c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:39.152962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702ed88-0783-4dbf-b8d4-3db46495389c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:18:39.154518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702ed88-0783-4dbf-b8d4-3db46495389c service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:18:39.165299 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1108/2245] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:18:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:18:46.041878 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944473f-5e20-4501-b5b2-3f513866b6d8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:46.042358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944473f-5e20-4501-b5b2-3f513866b6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:46.042461 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944473f-5e20-4501-b5b2-3f513866b6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:46.061794 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9944473f-5e20-4501-b5b2-3f513866b6d8 None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:18:46.062997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9944473f-5e20-4501-b5b2-3f513866b6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:46.117246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1138/2246] 10.222.0.30 () {58 vars in 967 bytes} [Thu Oct 6 08:18:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:18:46.139572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702ed88-0783-4dbf-b8d4-3db46495389c service nova] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:18:46.141002 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f702ed88-0783-4dbf-b8d4-3db46495389c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:18:46.211773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702ed88-0783-4dbf-b8d4-3db46495389c service nova] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:18:46.216696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2d7d0-c47b-4684-a96d-8c68c8aa878f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:18:46.217303 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2d7d0-c47b-4684-a96d-8c68c8aa878f service swift] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:18:46.217580 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2d7d0-c47b-4684-a96d-8c68c8aa878f service swift] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:18:46.217851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2d7d0-c47b-4684-a96d-8c68c8aa878f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:18:46.226335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2d7d0-c47b-4684-a96d-8c68c8aa878f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:18:46.227959 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2d7d0-c47b-4684-a96d-8c68c8aa878f service swift] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:18:46.236629 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1109/2247] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:18:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:18:56.109760 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=25 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:18:56.232829 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=60 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:19:14.138018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944473f-5e20-4501-b5b2-3f513866b6d8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:19:14.148801 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944473f-5e20-4501-b5b2-3f513866b6d8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:19:14.152388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82423e8-88f1-4f82-880a-54526efadc57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:19:14.152991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82423e8-88f1-4f82-880a-54526efadc57 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:19:14.153261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82423e8-88f1-4f82-880a-54526efadc57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:19:14.153529 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82423e8-88f1-4f82-880a-54526efadc57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:19:14.155429 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a82423e8-88f1-4f82-880a-54526efadc57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:19:14.166460 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82423e8-88f1-4f82-880a-54526efadc57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:19:14.167998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82423e8-88f1-4f82-880a-54526efadc57 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:19:14.176065 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1139/2248] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:19:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:19:15.749965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2d7d0-c47b-4684-a96d-8c68c8aa878f service swift] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:19:15.762232 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2d7d0-c47b-4684-a96d-8c68c8aa878f service swift] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:19:15.768143 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406f6c5-5586-45b0-97ec-60fe361d0e5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:19:15.768764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406f6c5-5586-45b0-97ec-60fe361d0e5f service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:19:15.769039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406f6c5-5586-45b0-97ec-60fe361d0e5f service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:19:15.769305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406f6c5-5586-45b0-97ec-60fe361d0e5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:19:15.771893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3406f6c5-5586-45b0-97ec-60fe361d0e5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:19:15.784022 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406f6c5-5586-45b0-97ec-60fe361d0e5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:19:15.785590 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406f6c5-5586-45b0-97ec-60fe361d0e5f service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:19:15.796651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1110/2249] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:19:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:19:24.172466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:19:25.792347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=6 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:19:26.940232 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82423e8-88f1-4f82-880a-54526efadc57 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:19:26.941544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a82423e8-88f1-4f82-880a-54526efadc57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:19:26.957308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82423e8-88f1-4f82-880a-54526efadc57 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:19:26.963471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be99bc-d1fd-45bd-9411-894d4c965ca8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:19:26.964058 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be99bc-d1fd-45bd-9411-894d4c965ca8 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:19:26.964328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be99bc-d1fd-45bd-9411-894d4c965ca8 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:19:26.964593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be99bc-d1fd-45bd-9411-894d4c965ca8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:19:26.966238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f3be99bc-d1fd-45bd-9411-894d4c965ca8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:19:27.025091 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be99bc-d1fd-45bd-9411-894d4c965ca8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:19:27.026694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be99bc-d1fd-45bd-9411-894d4c965ca8 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:19:27.035966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1140/2250] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:19:26 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:19:37.031634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=28 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:19:39.320111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406f6c5-5586-45b0-97ec-60fe361d0e5f service nova] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:19:39.337366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406f6c5-5586-45b0-97ec-60fe361d0e5f service nova] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:19:39.343508 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068bf2a-a88f-4075-9915-2e571cfbf55f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:19:39.344113 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068bf2a-a88f-4075-9915-2e571cfbf55f service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:19:39.344386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068bf2a-a88f-4075-9915-2e571cfbf55f service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:19:39.344680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068bf2a-a88f-4075-9915-2e571cfbf55f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:19:39.346391 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7068bf2a-a88f-4075-9915-2e571cfbf55f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:19:39.406698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7068bf2a-a88f-4075-9915-2e571cfbf55f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:19:39.408401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7068bf2a-a88f-4075-9915-2e571cfbf55f service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:19:39.418603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1111/2251] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:19:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:19:47.935780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556bdf1-1390-4416-8626-16e807d9bab0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:19:47.935780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556bdf1-1390-4416-8626-16e807d9bab0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:19:47.935780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556bdf1-1390-4416-8626-16e807d9bab0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:19:47.937019 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1141/2252] 10.222.0.30 () {54 vars in 921 bytes} [Thu Oct 6 08:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:19:47.967464 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87fd11-ca42-4cc0-a25d-f4bd751e3cc1 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:19:47.967995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87fd11-ca42-4cc0-a25d-f4bd751e3cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:19:47.968723 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87fd11-ca42-4cc0-a25d-f4bd751e3cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:19:47.991054 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-5f87fd11-ca42-4cc0-a25d-f4bd751e3cc1 None None] MFA Rules not processed for user `6074872303d3468fb566378fa34e8bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:19:47.992283 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5f87fd11-ca42-4cc0-a25d-f4bd751e3cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:19:48.058687 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1112/2253] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Oct 6 08:19:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:19:48.072163 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556bdf1-1390-4416-8626-16e807d9bab0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:19:48.073979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c556bdf1-1390-4416-8626-16e807d9bab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:19:48.094519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87fd11-ca42-4cc0-a25d-f4bd751e3cc1 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:19:48.095578 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556bdf1-1390-4416-8626-16e807d9bab0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:19:48.095950 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5f87fd11-ca42-4cc0-a25d-f4bd751e3cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:19:48.101461 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5431107-e04f-478a-a750-49e2523979a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6074872303d3468fb566378fa34e8bea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:19:48.102192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5431107-e04f-478a-a750-49e2523979a7 service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:19:48.102697 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5431107-e04f-478a-a750-49e2523979a7 service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:19:48.103176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5431107-e04f-478a-a750-49e2523979a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:19:48.113140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5431107-e04f-478a-a750-49e2523979a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:19:48.118398 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5431107-e04f-478a-a750-49e2523979a7 service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:19:48.130624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1142/2254] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:19:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:19:48.181998 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87fd11-ca42-4cc0-a25d-f4bd751e3cc1 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:19:48.186516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb295b0-54b6-4365-a6c1-1fc59f65afa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:19:48.187159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb295b0-54b6-4365-a6c1-1fc59f65afa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:19:48.187448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb295b0-54b6-4365-a6c1-1fc59f65afa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:19:48.188195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb295b0-54b6-4365-a6c1-1fc59f65afa9 admin admin] PATH_INFO: `/v3/users/fff5a88cba04475c9a0142156a2269f0` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:19:48.189306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb295b0-54b6-4365-a6c1-1fc59f65afa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fff5a88cba04475c9a0142156a2269f0)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:19:48.191343 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb295b0-54b6-4365-a6c1-1fc59f65afa9 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:19:48.430841 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:19:48.440407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9cb295b0-54b6-4365-a6c1-1fc59f65afa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fff5a88cba04475c9a0142156a2269f0'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:19:48.441112 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9cb295b0-54b6-4365-a6c1-1fc59f65afa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fff5a88cba04475c9a0142156a2269f0'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:19:48.453513 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9cb295b0-54b6-4365-a6c1-1fc59f65afa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fff5a88cba04475c9a0142156a2269f0'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:19:48.562869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9cb295b0-54b6-4365-a6c1-1fc59f65afa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fff5a88cba04475c9a0142156a2269f0'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:19:48.564154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9cb295b0-54b6-4365-a6c1-1fc59f65afa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fff5a88cba04475c9a0142156a2269f0'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:19:48.577415 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1113/2255] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:19:48 2022] DELETE /identity/v3/users/fff5a88cba04475c9a0142156a2269f0 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:19:48.816032 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5431107-e04f-478a-a750-49e2523979a7 service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:19:48.826801 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5431107-e04f-478a-a750-49e2523979a7 service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:19:48.887525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206d496-d03a-4410-af59-f30240cea1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:19:48.889716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206d496-d03a-4410-af59-f30240cea1a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:19:48.889716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206d496-d03a-4410-af59-f30240cea1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:19:48.889716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206d496-d03a-4410-af59-f30240cea1a9 admin admin] PATH_INFO: `/v3/projects/141863eb21404f229b8d624af2ff6743` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:19:48.890011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206d496-d03a-4410-af59-f30240cea1a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=141863eb21404f229b8d624af2ff6743)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:19:48.891868 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206d496-d03a-4410-af59-f30240cea1a9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:19:49.014060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1206d496-d03a-4410-af59-f30240cea1a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '141863eb21404f229b8d624af2ff6743', 'request_id': 'req-1206d496-d03a-4410-af59-f30240cea1a9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:19:49.020730 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1206d496-d03a-4410-af59-f30240cea1a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '141863eb21404f229b8d624af2ff6743', 'request_id': 'req-1206d496-d03a-4410-af59-f30240cea1a9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:19:49.106065 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1206d496-d03a-4410-af59-f30240cea1a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '141863eb21404f229b8d624af2ff6743', 'request_id': 'req-1206d496-d03a-4410-af59-f30240cea1a9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:19:49.117418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1206d496-d03a-4410-af59-f30240cea1a9 admin admin] The token cache is being invalidate because project 141863eb21404f229b8d624af2ff6743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:19:49.117709 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1206d496-d03a-4410-af59-f30240cea1a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:19:49.121551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1206d496-d03a-4410-af59-f30240cea1a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:19:49.121551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1143/2256] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:19:48 2022] DELETE /identity/v3/projects/141863eb21404f229b8d624af2ff6743 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:19:52.872118 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb295b0-54b6-4365-a6c1-1fc59f65afa9 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:19:52.873602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9cb295b0-54b6-4365-a6c1-1fc59f65afa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:19:52.946491 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb295b0-54b6-4365-a6c1-1fc59f65afa9 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:19:53.009328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4915022-eacb-4024-9bf0-f1268d5673fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:19:53.010282 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4915022-eacb-4024-9bf0-f1268d5673fb service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:19:53.010551 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4915022-eacb-4024-9bf0-f1268d5673fb service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:19:53.010815 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4915022-eacb-4024-9bf0-f1268d5673fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:19:53.012531 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a4915022-eacb-4024-9bf0-f1268d5673fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:19:53.059086 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:19:53.069293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4915022-eacb-4024-9bf0-f1268d5673fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:19:53.070913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4915022-eacb-4024-9bf0-f1268d5673fb service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:19:53.081000 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1114/2257] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:19:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:19:54.435491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab0528-c527-4d04-867f-79eb529974e8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:19:54.436043 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab0528-c527-4d04-867f-79eb529974e8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:19:54.436043 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab0528-c527-4d04-867f-79eb529974e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:19:54.456151 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-91ab0528-c527-4d04-867f-79eb529974e8 None None] MFA Rules not processed for user `85bdfc4b6ad443d68ebc0d6932ccad66`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:19:54.457353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-91ab0528-c527-4d04-867f-79eb529974e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:19:54.511103 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1144/2258] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:19:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:20:02.134720 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4915022-eacb-4024-9bf0-f1268d5673fb service nova] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:20:02.139875 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a4915022-eacb-4024-9bf0-f1268d5673fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:02.221228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4915022-eacb-4024-9bf0-f1268d5673fb service nova] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:20:02.228288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb6144-5a8f-4895-a350-926cfcb6531b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:20:02.228897 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb6144-5a8f-4895-a350-926cfcb6531b service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:20:02.229177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb6144-5a8f-4895-a350-926cfcb6531b service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:20:02.229446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb6144-5a8f-4895-a350-926cfcb6531b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:20:02.231369 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-06fb6144-5a8f-4895-a350-926cfcb6531b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:02.248077 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb6144-5a8f-4895-a350-926cfcb6531b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:20:02.250131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb6144-5a8f-4895-a350-926cfcb6531b service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:20:02.263226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1115/2259] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:20:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:20:04.504657 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:20:12.257904 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=33 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:20:12.683946 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2c050-2a6d-4e3f-bf41-afcb3679fdbb None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:20:12.684449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2c050-2a6d-4e3f-bf41-afcb3679fdbb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:20:12.684558 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2c050-2a6d-4e3f-bf41-afcb3679fdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:20:12.703433 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d5d2c050-2a6d-4e3f-bf41-afcb3679fdbb None None] MFA Rules not processed for user `9932c7fb57394d709a27023f3ae2677e`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:20:12.704655 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d5d2c050-2a6d-4e3f-bf41-afcb3679fdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:12.711783 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1145/2260] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:20:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:20:13.028255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb6144-5a8f-4895-a350-926cfcb6531b service nova] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:20:13.029812 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-06fb6144-5a8f-4895-a350-926cfcb6531b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:13.107023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb6144-5a8f-4895-a350-926cfcb6531b service nova] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:20:13.110607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7e8b8-7297-4dd2-9278-d294897002bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:20:13.111236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7e8b8-7297-4dd2-9278-d294897002bf service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:20:13.111504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7e8b8-7297-4dd2-9278-d294897002bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:20:13.115147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7e8b8-7297-4dd2-9278-d294897002bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:20:13.115147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-48f7e8b8-7297-4dd2-9278-d294897002bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:13.123778 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7e8b8-7297-4dd2-9278-d294897002bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:20:13.125192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7e8b8-7297-4dd2-9278-d294897002bf service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:20:13.133705 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1116/2261] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:20:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:20:14.595513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2c050-2a6d-4e3f-bf41-afcb3679fdbb None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:20:14.598132 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d5d2c050-2a6d-4e3f-bf41-afcb3679fdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:14.695843 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2c050-2a6d-4e3f-bf41-afcb3679fdbb None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:20:14.703320 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe99ea6-5a66-4e0c-b3a5-b5856eb6896c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6074872303d3468fb566378fa34e8bea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:20:14.704176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe99ea6-5a66-4e0c-b3a5-b5856eb6896c service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:20:14.723964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe99ea6-5a66-4e0c-b3a5-b5856eb6896c service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:20:14.723964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe99ea6-5a66-4e0c-b3a5-b5856eb6896c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:20:14.723964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bfe99ea6-5a66-4e0c-b3a5-b5856eb6896c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:14.723964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe99ea6-5a66-4e0c-b3a5-b5856eb6896c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:20:14.723964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe99ea6-5a66-4e0c-b3a5-b5856eb6896c service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:20:14.733413 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1146/2262] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:20:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:20:20.033340 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807d9e0-c581-4299-a3ee-eda72c02a420 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:20:20.033828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807d9e0-c581-4299-a3ee-eda72c02a420 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:20:20.033940 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807d9e0-c581-4299-a3ee-eda72c02a420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:20:20.053668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-b807d9e0-c581-4299-a3ee-eda72c02a420 None None] MFA Rules not processed for user `36b26040882a4e85b82b40a3c17d33bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:20:20.054786 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b807d9e0-c581-4299-a3ee-eda72c02a420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:20.110012 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1117/2263] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:20:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:20:24.728344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=35 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:20:26.802948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe99ea6-5a66-4e0c-b3a5-b5856eb6896c service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:20:26.804508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bfe99ea6-5a66-4e0c-b3a5-b5856eb6896c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:26.820571 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe99ea6-5a66-4e0c-b3a5-b5856eb6896c service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:20:26.826240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2911e9-bd26-4069-b503-47ed65927f64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:20:26.826843 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2911e9-bd26-4069-b503-47ed65927f64 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:20:26.827160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2911e9-bd26-4069-b503-47ed65927f64 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:20:26.827436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2911e9-bd26-4069-b503-47ed65927f64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:20:26.829162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7c2911e9-bd26-4069-b503-47ed65927f64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:26.842197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2911e9-bd26-4069-b503-47ed65927f64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:20:26.843834 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2911e9-bd26-4069-b503-47ed65927f64 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:20:26.854380 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1147/2264] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:20:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:20:30.102836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=60 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:20:33.505139 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807d9e0-c581-4299-a3ee-eda72c02a420 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:20:33.519287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807d9e0-c581-4299-a3ee-eda72c02a420 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:20:33.524990 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b5b3c-e686-4a7a-8c9c-083855ab8c95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:20:33.525599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b5b3c-e686-4a7a-8c9c-083855ab8c95 service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:20:33.525879 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b5b3c-e686-4a7a-8c9c-083855ab8c95 service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:20:33.526152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b5b3c-e686-4a7a-8c9c-083855ab8c95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:20:33.527930 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6d0b5b3c-e686-4a7a-8c9c-083855ab8c95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:33.601783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b5b3c-e686-4a7a-8c9c-083855ab8c95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:20:33.603768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b5b3c-e686-4a7a-8c9c-083855ab8c95 service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:20:33.616491 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1118/2265] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:20:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:20:36.849917 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:20:43.609317 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=28 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:20:44.962847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402d4cb-6ac9-4a48-a06a-5f4b05a19af6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:20:44.963319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402d4cb-6ac9-4a48-a06a-5f4b05a19af6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:20:44.963447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402d4cb-6ac9-4a48-a06a-5f4b05a19af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:20:44.993605 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0402d4cb-6ac9-4a48-a06a-5f4b05a19af6 None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:20:44.994789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0402d4cb-6ac9-4a48-a06a-5f4b05a19af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:45.055420 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1148/2266] 10.222.0.30 () {58 vars in 967 bytes} [Thu Oct 6 08:20:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:20:45.217604 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b5b3c-e686-4a7a-8c9c-083855ab8c95 service nova] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:20:45.219086 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6d0b5b3c-e686-4a7a-8c9c-083855ab8c95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:45.286553 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b5b3c-e686-4a7a-8c9c-083855ab8c95 service nova] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:20:45.291587 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e9a91-67a7-4131-a84c-53ac1b104c1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:20:45.292185 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e9a91-67a7-4131-a84c-53ac1b104c1a service swift] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:20:45.292458 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e9a91-67a7-4131-a84c-53ac1b104c1a service swift] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:20:45.292728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e9a91-67a7-4131-a84c-53ac1b104c1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:20:45.301376 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e9a91-67a7-4131-a84c-53ac1b104c1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:20:45.303007 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e9a91-67a7-4131-a84c-53ac1b104c1a service swift] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:20:45.310989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1119/2267] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:20:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:20:46.181557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402d4cb-6ac9-4a48-a06a-5f4b05a19af6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:20:46.182990 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0402d4cb-6ac9-4a48-a06a-5f4b05a19af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:46.194913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402d4cb-6ac9-4a48-a06a-5f4b05a19af6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:20:46.198476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3c7ed-4274-4baa-927f-36447d20ba17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:20:46.199106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3c7ed-4274-4baa-927f-36447d20ba17 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:20:46.199378 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3c7ed-4274-4baa-927f-36447d20ba17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:20:46.199643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3c7ed-4274-4baa-927f-36447d20ba17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:20:46.201306 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d9c3c7ed-4274-4baa-927f-36447d20ba17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:46.283479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3c7ed-4274-4baa-927f-36447d20ba17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:20:46.284916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3c7ed-4274-4baa-927f-36447d20ba17 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:20:46.304667 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1149/2268] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:20:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:20:53.576676 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e9a91-67a7-4131-a84c-53ac1b104c1a service swift] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:20:53.578303 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-918e9a91-67a7-4131-a84c-53ac1b104c1a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:53.592575 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e9a91-67a7-4131-a84c-53ac1b104c1a service swift] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:20:53.597221 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8418b-b905-4fac-9d82-1c1231167862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:20:53.597844 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8418b-b905-4fac-9d82-1c1231167862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:20:53.598119 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8418b-b905-4fac-9d82-1c1231167862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:20:53.598392 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8418b-b905-4fac-9d82-1c1231167862 admin admin] PATH_INFO: `/v3/users/e767c7081875428382af11d1ff475a20` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:20:53.599533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8418b-b905-4fac-9d82-1c1231167862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e767c7081875428382af11d1ff475a20)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:20:53.601495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8418b-b905-4fac-9d82-1c1231167862 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:20:53.740006 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-32f8418b-b905-4fac-9d82-1c1231167862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e767c7081875428382af11d1ff475a20'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:53.745165 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-32f8418b-b905-4fac-9d82-1c1231167862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e767c7081875428382af11d1ff475a20'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:53.753077 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-32f8418b-b905-4fac-9d82-1c1231167862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e767c7081875428382af11d1ff475a20'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:53.818130 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-32f8418b-b905-4fac-9d82-1c1231167862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e767c7081875428382af11d1ff475a20'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:53.818749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-32f8418b-b905-4fac-9d82-1c1231167862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e767c7081875428382af11d1ff475a20'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:53.831083 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1120/2269] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:20:53 2022] DELETE /identity/v3/users/e767c7081875428382af11d1ff475a20 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:20:54.100803 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3c7ed-4274-4baa-927f-36447d20ba17 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:20:54.110721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3c7ed-4274-4baa-927f-36447d20ba17 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:20:54.176983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691be3d-5b32-4c9f-83cb-f518c2b356fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:20:54.177624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691be3d-5b32-4c9f-83cb-f518c2b356fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:20:54.177991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691be3d-5b32-4c9f-83cb-f518c2b356fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:20:54.178208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691be3d-5b32-4c9f-83cb-f518c2b356fb admin admin] PATH_INFO: `/v3/projects/9b4d6dd628464c05b707fee1dd32f04a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:20:54.179368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691be3d-5b32-4c9f-83cb-f518c2b356fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b4d6dd628464c05b707fee1dd32f04a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:20:54.181288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691be3d-5b32-4c9f-83cb-f518c2b356fb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:20:54.183978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:20:54.351129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2691be3d-5b32-4c9f-83cb-f518c2b356fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b4d6dd628464c05b707fee1dd32f04a', 'request_id': 'req-2691be3d-5b32-4c9f-83cb-f518c2b356fb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:54.357352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2691be3d-5b32-4c9f-83cb-f518c2b356fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b4d6dd628464c05b707fee1dd32f04a', 'request_id': 'req-2691be3d-5b32-4c9f-83cb-f518c2b356fb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:54.407508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2691be3d-5b32-4c9f-83cb-f518c2b356fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b4d6dd628464c05b707fee1dd32f04a', 'request_id': 'req-2691be3d-5b32-4c9f-83cb-f518c2b356fb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:54.418075 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2691be3d-5b32-4c9f-83cb-f518c2b356fb admin admin] The token cache is being invalidate because project 9b4d6dd628464c05b707fee1dd32f04a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:20:54.418624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2691be3d-5b32-4c9f-83cb-f518c2b356fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:54.419858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2691be3d-5b32-4c9f-83cb-f518c2b356fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:54.421367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1150/2270] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:20:54 2022] DELETE /identity/v3/projects/9b4d6dd628464c05b707fee1dd32f04a => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:20:54.426175 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8418b-b905-4fac-9d82-1c1231167862 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:20:54.427652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-32f8418b-b905-4fac-9d82-1c1231167862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:54.441748 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8418b-b905-4fac-9d82-1c1231167862 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:20:54.496990 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a59fa-0075-4d6f-a389-c2de2dce7b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:20:54.497607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a59fa-0075-4d6f-a389-c2de2dce7b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:20:54.497889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a59fa-0075-4d6f-a389-c2de2dce7b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:20:54.498151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a59fa-0075-4d6f-a389-c2de2dce7b16 admin admin] PATH_INFO: `/v3/users/b0e5abb88fa74657a025bef294b74b91` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:20:54.499285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a59fa-0075-4d6f-a389-c2de2dce7b16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0e5abb88fa74657a025bef294b74b91)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:20:54.501191 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a59fa-0075-4d6f-a389-c2de2dce7b16 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:20:54.504027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:20:54.745994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3d2a59fa-0075-4d6f-a389-c2de2dce7b16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0e5abb88fa74657a025bef294b74b91'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:54.746646 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3d2a59fa-0075-4d6f-a389-c2de2dce7b16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0e5abb88fa74657a025bef294b74b91'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:54.758487 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3d2a59fa-0075-4d6f-a389-c2de2dce7b16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0e5abb88fa74657a025bef294b74b91'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:54.824297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3d2a59fa-0075-4d6f-a389-c2de2dce7b16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0e5abb88fa74657a025bef294b74b91'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:54.824922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3d2a59fa-0075-4d6f-a389-c2de2dce7b16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0e5abb88fa74657a025bef294b74b91'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:54.837567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1121/2271] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:20:54 2022] DELETE /identity/v3/users/b0e5abb88fa74657a025bef294b74b91 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:20:55.077251 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691be3d-5b32-4c9f-83cb-f518c2b356fb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:20:55.078853 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2691be3d-5b32-4c9f-83cb-f518c2b356fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:20:55.099917 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691be3d-5b32-4c9f-83cb-f518c2b356fb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:20:55.156908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2efdd-efe5-4852-92de-693a5eb5da74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:20:55.157533 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2efdd-efe5-4852-92de-693a5eb5da74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:20:55.157804 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2efdd-efe5-4852-92de-693a5eb5da74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:20:55.158104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2efdd-efe5-4852-92de-693a5eb5da74 admin admin] PATH_INFO: `/v3/projects/59fe95b503f5440bae2ed05569214c29` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:20:55.159891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2efdd-efe5-4852-92de-693a5eb5da74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59fe95b503f5440bae2ed05569214c29)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:20:55.161827 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2efdd-efe5-4852-92de-693a5eb5da74 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:20:55.271321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cce2efdd-efe5-4852-92de-693a5eb5da74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59fe95b503f5440bae2ed05569214c29', 'request_id': 'req-cce2efdd-efe5-4852-92de-693a5eb5da74'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:55.277413 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cce2efdd-efe5-4852-92de-693a5eb5da74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59fe95b503f5440bae2ed05569214c29', 'request_id': 'req-cce2efdd-efe5-4852-92de-693a5eb5da74'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:55.487364 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cce2efdd-efe5-4852-92de-693a5eb5da74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59fe95b503f5440bae2ed05569214c29', 'request_id': 'req-cce2efdd-efe5-4852-92de-693a5eb5da74'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:55.498537 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cce2efdd-efe5-4852-92de-693a5eb5da74 admin admin] The token cache is being invalidate because project 59fe95b503f5440bae2ed05569214c29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:20:55.498823 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cce2efdd-efe5-4852-92de-693a5eb5da74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:55.499471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cce2efdd-efe5-4852-92de-693a5eb5da74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:20:55.500969 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1151/2272] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:20:55 2022] DELETE /identity/v3/projects/59fe95b503f5440bae2ed05569214c29 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:04.834021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:05.495254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=74,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:14.467246 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6c213-6c52-4515-8066-c0203fe021ac None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:14.467767 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6c213-6c52-4515-8066-c0203fe021ac None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:14.467872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6c213-6c52-4515-8066-c0203fe021ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:14.487641 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-4da6c213-6c52-4515-8066-c0203fe021ac None None] MFA Rules not processed for user `cb73bbd331754280a2a0cd64290ec3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:21:14.488840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4da6c213-6c52-4515-8066-c0203fe021ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:14.544071 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1122/2273] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:21:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:21:24.536798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=25 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:31.154666 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2efdd-efe5-4852-92de-693a5eb5da74 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:31.156763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cce2efdd-efe5-4852-92de-693a5eb5da74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:31.221746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2efdd-efe5-4852-92de-693a5eb5da74 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:31.225815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd06ef-ecbb-4288-a6d7-a91619cb5081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:31.226498 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd06ef-ecbb-4288-a6d7-a91619cb5081 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:31.226849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd06ef-ecbb-4288-a6d7-a91619cb5081 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:31.227530 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd06ef-ecbb-4288-a6d7-a91619cb5081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:31.229314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9ebd06ef-ecbb-4288-a6d7-a91619cb5081 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:31.298820 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd06ef-ecbb-4288-a6d7-a91619cb5081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:31.300208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd06ef-ecbb-4288-a6d7-a91619cb5081 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:31.311452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1152/2274] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:21:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:21:35.483172 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6c213-6c52-4515-8066-c0203fe021ac None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:35.484694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4da6c213-6c52-4515-8066-c0203fe021ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:35.500658 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6c213-6c52-4515-8066-c0203fe021ac None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:35.505273 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111718e-6ea3-461c-beb7-daf7d0351780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:35.508519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111718e-6ea3-461c-beb7-daf7d0351780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:35.508519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111718e-6ea3-461c-beb7-daf7d0351780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:35.508519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111718e-6ea3-461c-beb7-daf7d0351780 admin admin] PATH_INFO: `/v3/users/85bdfc4b6ad443d68ebc0d6932ccad66` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:35.508519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111718e-6ea3-461c-beb7-daf7d0351780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85bdfc4b6ad443d68ebc0d6932ccad66)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:35.509641 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111718e-6ea3-461c-beb7-daf7d0351780 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:35.685559 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-4111718e-6ea3-461c-beb7-daf7d0351780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85bdfc4b6ad443d68ebc0d6932ccad66'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:35.686212 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-4111718e-6ea3-461c-beb7-daf7d0351780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85bdfc4b6ad443d68ebc0d6932ccad66'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:35.698225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-4111718e-6ea3-461c-beb7-daf7d0351780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85bdfc4b6ad443d68ebc0d6932ccad66'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:35.773417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-4111718e-6ea3-461c-beb7-daf7d0351780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85bdfc4b6ad443d68ebc0d6932ccad66'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:35.774049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-4111718e-6ea3-461c-beb7-daf7d0351780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85bdfc4b6ad443d68ebc0d6932ccad66'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:35.787576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1123/2275] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:21:35 2022] DELETE /identity/v3/users/85bdfc4b6ad443d68ebc0d6932ccad66 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:36.020844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd06ef-ecbb-4288-a6d7-a91619cb5081 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:36.033055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd06ef-ecbb-4288-a6d7-a91619cb5081 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:36.104378 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf90df-c779-48c4-ae2c-0f00eac27820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:36.106811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf90df-c779-48c4-ae2c-0f00eac27820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:36.106811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf90df-c779-48c4-ae2c-0f00eac27820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:36.106811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf90df-c779-48c4-ae2c-0f00eac27820 admin admin] PATH_INFO: `/v3/projects/d5f023bd0dcf41558003e085c1dddf30` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:36.107643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf90df-c779-48c4-ae2c-0f00eac27820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5f023bd0dcf41558003e085c1dddf30)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:36.109570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf90df-c779-48c4-ae2c-0f00eac27820 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:36.225864 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90bf90df-c779-48c4-ae2c-0f00eac27820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5f023bd0dcf41558003e085c1dddf30', 'request_id': 'req-90bf90df-c779-48c4-ae2c-0f00eac27820'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:36.227841 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:36.232868 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90bf90df-c779-48c4-ae2c-0f00eac27820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5f023bd0dcf41558003e085c1dddf30', 'request_id': 'req-90bf90df-c779-48c4-ae2c-0f00eac27820'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:36.333037 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90bf90df-c779-48c4-ae2c-0f00eac27820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5f023bd0dcf41558003e085c1dddf30', 'request_id': 'req-90bf90df-c779-48c4-ae2c-0f00eac27820'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:36.344301 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90bf90df-c779-48c4-ae2c-0f00eac27820 admin admin] The token cache is being invalidate because project d5f023bd0dcf41558003e085c1dddf30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:21:36.346139 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90bf90df-c779-48c4-ae2c-0f00eac27820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:36.346139 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90bf90df-c779-48c4-ae2c-0f00eac27820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:36.346804 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1153/2276] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:21:36 2022] DELETE /identity/v3/projects/d5f023bd0dcf41558003e085c1dddf30 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:36.351701 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f5a31-8289-4e5b-bc5a-0a70ec4144ff None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:36.351984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f5a31-8289-4e5b-bc5a-0a70ec4144ff None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:36.352284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f5a31-8289-4e5b-bc5a-0a70ec4144ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:36.409811 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-5c3f5a31-8289-4e5b-bc5a-0a70ec4144ff None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:21:36.411039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5c3f5a31-8289-4e5b-bc5a-0a70ec4144ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:36.469215 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1124/2277] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:21:36.476972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41647a25-3d9b-4f49-a1b3-721d0961bff5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:36.477253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41647a25-3d9b-4f49-a1b3-721d0961bff5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:36.477524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41647a25-3d9b-4f49-a1b3-721d0961bff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:36.496901 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-41647a25-3d9b-4f49-a1b3-721d0961bff5 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:21:36.498104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-41647a25-3d9b-4f49-a1b3-721d0961bff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:36.506448 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1154/2278] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:21:36.510343 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f5a31-8289-4e5b-bc5a-0a70ec4144ff None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:36.520918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f5a31-8289-4e5b-bc5a-0a70ec4144ff None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:36.525580 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dcdb1f-7694-4cbc-8534-ad2a30cad9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:36.526173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dcdb1f-7694-4cbc-8534-ad2a30cad9bf admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:36.526449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dcdb1f-7694-4cbc-8534-ad2a30cad9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:36.526716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dcdb1f-7694-4cbc-8534-ad2a30cad9bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:36.527960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dcdb1f-7694-4cbc-8534-ad2a30cad9bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:21:36.528265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dcdb1f-7694-4cbc-8534-ad2a30cad9bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:36.529865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dcdb1f-7694-4cbc-8534-ad2a30cad9bf admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:36.541302 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1125/2279] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:21:36.546154 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41647a25-3d9b-4f49-a1b3-721d0961bff5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:36.555299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41647a25-3d9b-4f49-a1b3-721d0961bff5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:36.560519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a223428-f4d8-4afc-8802-55a632761df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:36.561114 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a223428-f4d8-4afc-8802-55a632761df3 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:36.561389 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a223428-f4d8-4afc-8802-55a632761df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:36.561656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a223428-f4d8-4afc-8802-55a632761df3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:36.562280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a223428-f4d8-4afc-8802-55a632761df3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:36.564194 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a223428-f4d8-4afc-8802-55a632761df3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:36.686986 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1155/2280] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:21:36 2022] POST /identity/v3/projects => generated 369 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:21:36.688256 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dcdb1f-7694-4cbc-8534-ad2a30cad9bf admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:36.698404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dcdb1f-7694-4cbc-8534-ad2a30cad9bf admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:36.724988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:36.755483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970915c4-a309-4b98-ba17-cc3799f93b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:36.756140 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970915c4-a309-4b98-ba17-cc3799f93b2f admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:36.756422 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970915c4-a309-4b98-ba17-cc3799f93b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:36.756896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970915c4-a309-4b98-ba17-cc3799f93b2f admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:36.758002 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970915c4-a309-4b98-ba17-cc3799f93b2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:36.759980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970915c4-a309-4b98-ba17-cc3799f93b2f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:36.871576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1126/2281] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:21:36 2022] POST /identity/v3/users => generated 285 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:21:36.875724 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a223428-f4d8-4afc-8802-55a632761df3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:36.885905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a223428-f4d8-4afc-8802-55a632761df3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:36.891188 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00340d33-be70-4949-a6be-2f7678ea8440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:36.891848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00340d33-be70-4949-a6be-2f7678ea8440 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:36.892115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00340d33-be70-4949-a6be-2f7678ea8440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:36.892505 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00340d33-be70-4949-a6be-2f7678ea8440 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:36.893241 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00340d33-be70-4949-a6be-2f7678ea8440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:36.895364 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00340d33-be70-4949-a6be-2f7678ea8440 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:36.908253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1156/2282] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:21:36.912828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970915c4-a309-4b98-ba17-cc3799f93b2f admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:36.925318 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970915c4-a309-4b98-ba17-cc3799f93b2f admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:36.930218 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba22e5-95b1-4daa-a42b-ca60f1221776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:36.930847 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba22e5-95b1-4daa-a42b-ca60f1221776 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:36.931147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba22e5-95b1-4daa-a42b-ca60f1221776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:36.931419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba22e5-95b1-4daa-a42b-ca60f1221776 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:36.932124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba22e5-95b1-4daa-a42b-ca60f1221776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:36.934354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba22e5-95b1-4daa-a42b-ca60f1221776 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:36.944404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1127/2283] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:21:36.949448 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00340d33-be70-4949-a6be-2f7678ea8440 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:36.959842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00340d33-be70-4949-a6be-2f7678ea8440 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:36.967288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c190b37-a309-451a-a38a-da6fbce66d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:36.967288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c190b37-a309-451a-a38a-da6fbce66d2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:36.967288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c190b37-a309-451a-a38a-da6fbce66d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:36.967288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c190b37-a309-451a-a38a-da6fbce66d2a admin admin] PATH_INFO: `/v3/projects/be9c65c5e7454d2eba6426d1cf6c9cdb/users/b4860ccc40624033ad8e26cf23a7f0ef/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:36.981319 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c190b37-a309-451a-a38a-da6fbce66d2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=be9c65c5e7454d2eba6426d1cf6c9cdb, user_id=b4860ccc40624033ad8e26cf23a7f0ef, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:36.984210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c190b37-a309-451a-a38a-da6fbce66d2a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:37.047245 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1157/2284] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:21:36 2022] PUT /identity/v3/projects/be9c65c5e7454d2eba6426d1cf6c9cdb/users/b4860ccc40624033ad8e26cf23a7f0ef/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:21:37.058887 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e81893-7a56-437b-b655-ff8fd5e8ab43 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:37.059210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e81893-7a56-437b-b655-ff8fd5e8ab43 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:37.059479 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e81893-7a56-437b-b655-ff8fd5e8ab43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:37.087993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-f2e81893-7a56-437b-b655-ff8fd5e8ab43 None None] MFA Rules not processed for user `b4860ccc40624033ad8e26cf23a7f0ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:21:37.089251 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f2e81893-7a56-437b-b655-ff8fd5e8ab43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:37.156238 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1128/2285] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:21:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:21:37.160714 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c190b37-a309-451a-a38a-da6fbce66d2a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:37.222720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c190b37-a309-451a-a38a-da6fbce66d2a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:37.227225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41faf2b9-214d-4f32-bbad-078a5ee2340d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:37.227815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41faf2b9-214d-4f32-bbad-078a5ee2340d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:37.228089 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41faf2b9-214d-4f32-bbad-078a5ee2340d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:37.228355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41faf2b9-214d-4f32-bbad-078a5ee2340d admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:37.228980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41faf2b9-214d-4f32-bbad-078a5ee2340d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:37.231441 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41faf2b9-214d-4f32-bbad-078a5ee2340d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:37.334922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1158/2286] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:21:37 2022] POST /identity/v3/projects => generated 367 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:21:37.338997 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e81893-7a56-437b-b655-ff8fd5e8ab43 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:37.350104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e81893-7a56-437b-b655-ff8fd5e8ab43 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:37.406369 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619fec69-c758-43d0-9094-c3729ebc372e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:37.406994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fec69-c758-43d0-9094-c3729ebc372e admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:37.407274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fec69-c758-43d0-9094-c3729ebc372e admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:37.407544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fec69-c758-43d0-9094-c3729ebc372e admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:37.408164 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619fec69-c758-43d0-9094-c3729ebc372e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:37.410104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619fec69-c758-43d0-9094-c3729ebc372e admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:37.498355 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1129/2287] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:21:37 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:21:37.502324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41faf2b9-214d-4f32-bbad-078a5ee2340d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:37.513701 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41faf2b9-214d-4f32-bbad-078a5ee2340d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:37.518471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a1956-2a99-431f-8dce-c89ce0c4ff69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:37.519206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a1956-2a99-431f-8dce-c89ce0c4ff69 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:37.519501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a1956-2a99-431f-8dce-c89ce0c4ff69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:37.519773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a1956-2a99-431f-8dce-c89ce0c4ff69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:37.520416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a1956-2a99-431f-8dce-c89ce0c4ff69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:37.522012 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a1956-2a99-431f-8dce-c89ce0c4ff69 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:37.533619 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1159/2288] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:21:37.537750 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619fec69-c758-43d0-9094-c3729ebc372e admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:37.548687 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619fec69-c758-43d0-9094-c3729ebc372e admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:37.553506 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933822f8-80f1-4097-9566-09e74cb6a979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:37.554161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933822f8-80f1-4097-9566-09e74cb6a979 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:37.554439 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933822f8-80f1-4097-9566-09e74cb6a979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:37.554711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933822f8-80f1-4097-9566-09e74cb6a979 admin admin] PATH_INFO: `/v3/projects/874c8bdd54ca4a438b0a0728e5ece490/users/83501845e9754f15b22a73bafe002d18/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:37.569474 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933822f8-80f1-4097-9566-09e74cb6a979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=874c8bdd54ca4a438b0a0728e5ece490, user_id=83501845e9754f15b22a73bafe002d18, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:37.572196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933822f8-80f1-4097-9566-09e74cb6a979 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:37.635573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1130/2289] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:21:37 2022] PUT /identity/v3/projects/874c8bdd54ca4a438b0a0728e5ece490/users/83501845e9754f15b22a73bafe002d18/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:37.639892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a1956-2a99-431f-8dce-c89ce0c4ff69 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:37.649676 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a1956-2a99-431f-8dce-c89ce0c4ff69 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:37.690531 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:37.706248 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb57d86-34c1-4fa0-ab6d-4e171684f9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:37.706842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb57d86-34c1-4fa0-ab6d-4e171684f9fe admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:37.707147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb57d86-34c1-4fa0-ab6d-4e171684f9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:37.707412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb57d86-34c1-4fa0-ab6d-4e171684f9fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:37.708066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb57d86-34c1-4fa0-ab6d-4e171684f9fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:37.709647 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb57d86-34c1-4fa0-ab6d-4e171684f9fe admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:37.720459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1160/2290] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:21:37.728131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933822f8-80f1-4097-9566-09e74cb6a979 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:37.734474 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:37.739711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933822f8-80f1-4097-9566-09e74cb6a979 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:37.744478 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad59bed-6a77-468f-8446-d25589149afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:37.745130 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad59bed-6a77-468f-8446-d25589149afb admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:37.745406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad59bed-6a77-468f-8446-d25589149afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:37.745764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad59bed-6a77-468f-8446-d25589149afb admin admin] PATH_INFO: `/v3/projects/874c8bdd54ca4a438b0a0728e5ece490/users/83501845e9754f15b22a73bafe002d18/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:37.747899 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad59bed-6a77-468f-8446-d25589149afb admin admin] RBAC: Authorizing `identity:create_grant(project_id=874c8bdd54ca4a438b0a0728e5ece490, user_id=83501845e9754f15b22a73bafe002d18, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:37.754194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad59bed-6a77-468f-8446-d25589149afb admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:37.760987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1131/2291] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:21:37 2022] PUT /identity/v3/projects/874c8bdd54ca4a438b0a0728e5ece490/users/83501845e9754f15b22a73bafe002d18/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:37.771260 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcb15d-74ed-45b9-a467-ae07d9cfda98 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:37.771568 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcb15d-74ed-45b9-a467-ae07d9cfda98 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:37.771839 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcb15d-74ed-45b9-a467-ae07d9cfda98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:37.801291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-89fcb15d-74ed-45b9-a467-ae07d9cfda98 None None] MFA Rules not processed for user `83501845e9754f15b22a73bafe002d18`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:21:37.802487 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-89fcb15d-74ed-45b9-a467-ae07d9cfda98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:37.869853 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1161/2292] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:21:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:21:37.881594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad59bed-6a77-468f-8446-d25589149afb admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:37.883313 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1ad59bed-6a77-468f-8446-d25589149afb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:37.961462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad59bed-6a77-468f-8446-d25589149afb admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:37.965794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8e920-64ce-4edf-8d28-c3dda5d33f18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:37.966388 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8e920-64ce-4edf-8d28-c3dda5d33f18 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:37.966667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8e920-64ce-4edf-8d28-c3dda5d33f18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:37.966959 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8e920-64ce-4edf-8d28-c3dda5d33f18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:37.975882 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8e920-64ce-4edf-8d28-c3dda5d33f18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:37.977306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8e920-64ce-4edf-8d28-c3dda5d33f18 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:37.986708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1132/2293] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:21:37 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:21:38.119718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcb15d-74ed-45b9-a467-ae07d9cfda98 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:38.121736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-89fcb15d-74ed-45b9-a467-ae07d9cfda98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:38.133493 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcb15d-74ed-45b9-a467-ae07d9cfda98 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:38.138337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d0e93d-3ab7-428d-bc56-9cf290d8200d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:38.138933 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0e93d-3ab7-428d-bc56-9cf290d8200d service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:38.139237 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0e93d-3ab7-428d-bc56-9cf290d8200d service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:38.139510 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0e93d-3ab7-428d-bc56-9cf290d8200d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:38.205561 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d0e93d-3ab7-428d-bc56-9cf290d8200d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:38.207510 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d0e93d-3ab7-428d-bc56-9cf290d8200d service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:38.222604 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1162/2294] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:21:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:21:42.892229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8e920-64ce-4edf-8d28-c3dda5d33f18 service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:42.954335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8e920-64ce-4edf-8d28-c3dda5d33f18 service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:42.958921 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8a68b-a88d-489b-9472-c62f155a77c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:42.959566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8a68b-a88d-489b-9472-c62f155a77c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:42.959845 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8a68b-a88d-489b-9472-c62f155a77c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:42.960115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8a68b-a88d-489b-9472-c62f155a77c9 admin admin] PATH_INFO: `/v3/users/b4860ccc40624033ad8e26cf23a7f0ef` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:42.961226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8a68b-a88d-489b-9472-c62f155a77c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4860ccc40624033ad8e26cf23a7f0ef)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:42.963191 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8a68b-a88d-489b-9472-c62f155a77c9 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:43.175712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-22b8a68b-a88d-489b-9472-c62f155a77c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4860ccc40624033ad8e26cf23a7f0ef'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:43.176373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-22b8a68b-a88d-489b-9472-c62f155a77c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4860ccc40624033ad8e26cf23a7f0ef'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:43.189087 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-22b8a68b-a88d-489b-9472-c62f155a77c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4860ccc40624033ad8e26cf23a7f0ef'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:43.274962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-22b8a68b-a88d-489b-9472-c62f155a77c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4860ccc40624033ad8e26cf23a7f0ef'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:43.275582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-22b8a68b-a88d-489b-9472-c62f155a77c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4860ccc40624033ad8e26cf23a7f0ef'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:43.278024 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:43.288865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1133/2295] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:21:42 2022] DELETE /identity/v3/users/b4860ccc40624033ad8e26cf23a7f0ef => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:43.294314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14dc0c-cbbf-40a0-98e9-9b86dedba0b0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:43.294767 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14dc0c-cbbf-40a0-98e9-9b86dedba0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:43.294885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14dc0c-cbbf-40a0-98e9-9b86dedba0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:43.316208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2d14dc0c-cbbf-40a0-98e9-9b86dedba0b0 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:21:43.317409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2d14dc0c-cbbf-40a0-98e9-9b86dedba0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:43.499179 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1163/2296] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:21:43.509265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8a68b-a88d-489b-9472-c62f155a77c9 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:43.510666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-22b8a68b-a88d-489b-9472-c62f155a77c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:43.567586 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8a68b-a88d-489b-9472-c62f155a77c9 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:43.571138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1d3c5-776f-44f8-b4ba-dc55b9304859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:43.571729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1d3c5-776f-44f8-b4ba-dc55b9304859 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:43.572001 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1d3c5-776f-44f8-b4ba-dc55b9304859 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:43.572269 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1d3c5-776f-44f8-b4ba-dc55b9304859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:43.581440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1d3c5-776f-44f8-b4ba-dc55b9304859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:43.582865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1d3c5-776f-44f8-b4ba-dc55b9304859 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:43.593120 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1134/2297] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:21:43.989314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14dc0c-cbbf-40a0-98e9-9b86dedba0b0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:43.990752 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2d14dc0c-cbbf-40a0-98e9-9b86dedba0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:43.998576 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:44.003876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14dc0c-cbbf-40a0-98e9-9b86dedba0b0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:44.008561 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:44.009192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:44.009468 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:44.009729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f admin admin] PATH_INFO: `/v3/projects/be9c65c5e7454d2eba6426d1cf6c9cdb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:44.010843 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f admin admin] RBAC: Authorizing `identity:delete_project(project_id=be9c65c5e7454d2eba6426d1cf6c9cdb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:44.014273 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:44.098239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be9c65c5e7454d2eba6426d1cf6c9cdb', 'request_id': 'req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:44.104643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be9c65c5e7454d2eba6426d1cf6c9cdb', 'request_id': 'req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:44.184736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be9c65c5e7454d2eba6426d1cf6c9cdb', 'request_id': 'req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:44.196112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f admin admin] The token cache is being invalidate because project be9c65c5e7454d2eba6426d1cf6c9cdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:21:44.196398 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:44.197104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:44.199220 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1164/2298] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:21:43 2022] DELETE /identity/v3/projects/be9c65c5e7454d2eba6426d1cf6c9cdb => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:44.203408 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1d3c5-776f-44f8-b4ba-dc55b9304859 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:44.213141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1d3c5-776f-44f8-b4ba-dc55b9304859 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:44.270644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e71f3-198d-445b-9728-c3705ebcc323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:44.270644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e71f3-198d-445b-9728-c3705ebcc323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:44.270644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e71f3-198d-445b-9728-c3705ebcc323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:44.270644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e71f3-198d-445b-9728-c3705ebcc323 admin admin] PATH_INFO: `/v3/users/83501845e9754f15b22a73bafe002d18` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:44.271937 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e71f3-198d-445b-9728-c3705ebcc323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83501845e9754f15b22a73bafe002d18)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:44.273877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e71f3-198d-445b-9728-c3705ebcc323 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:44.438001 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9c3e71f3-198d-445b-9728-c3705ebcc323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83501845e9754f15b22a73bafe002d18'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:44.438640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9c3e71f3-198d-445b-9728-c3705ebcc323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83501845e9754f15b22a73bafe002d18'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:44.451601 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9c3e71f3-198d-445b-9728-c3705ebcc323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83501845e9754f15b22a73bafe002d18'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:44.527455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9c3e71f3-198d-445b-9728-c3705ebcc323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83501845e9754f15b22a73bafe002d18'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:44.528082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-9c3e71f3-198d-445b-9728-c3705ebcc323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83501845e9754f15b22a73bafe002d18'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:44.540704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1135/2299] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:21:44 2022] DELETE /identity/v3/users/83501845e9754f15b22a73bafe002d18 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:44.916328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:44.918895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:44.936440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6f976-1a93-4ac2-999b-4ecdcb3a675f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:44.993918 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352833d-5a12-415a-ad68-b26283aecf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:44.994546 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352833d-5a12-415a-ad68-b26283aecf5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:44.994861 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352833d-5a12-415a-ad68-b26283aecf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:44.995122 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352833d-5a12-415a-ad68-b26283aecf5d admin admin] PATH_INFO: `/v3/projects/874c8bdd54ca4a438b0a0728e5ece490` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:45.009944 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352833d-5a12-415a-ad68-b26283aecf5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=874c8bdd54ca4a438b0a0728e5ece490)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:45.009944 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352833d-5a12-415a-ad68-b26283aecf5d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:45.109834 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d352833d-5a12-415a-ad68-b26283aecf5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '874c8bdd54ca4a438b0a0728e5ece490', 'request_id': 'req-d352833d-5a12-415a-ad68-b26283aecf5d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:45.115832 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d352833d-5a12-415a-ad68-b26283aecf5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '874c8bdd54ca4a438b0a0728e5ece490', 'request_id': 'req-d352833d-5a12-415a-ad68-b26283aecf5d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:45.178743 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d352833d-5a12-415a-ad68-b26283aecf5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '874c8bdd54ca4a438b0a0728e5ece490', 'request_id': 'req-d352833d-5a12-415a-ad68-b26283aecf5d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:45.190344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d352833d-5a12-415a-ad68-b26283aecf5d admin admin] The token cache is being invalidate because project 874c8bdd54ca4a438b0a0728e5ece490 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:21:45.190622 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d352833d-5a12-415a-ad68-b26283aecf5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:45.191280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d352833d-5a12-415a-ad68-b26283aecf5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:45.192738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1165/2300] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:21:44 2022] DELETE /identity/v3/projects/874c8bdd54ca4a438b0a0728e5ece490 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:45.197591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4d355-53da-400e-b8d6-8bf9462f9ae4 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:45.197881 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4d355-53da-400e-b8d6-8bf9462f9ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:45.198154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4d355-53da-400e-b8d6-8bf9462f9ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:45.218293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-c9b4d355-53da-400e-b8d6-8bf9462f9ae4 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:21:45.219512 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-c9b4d355-53da-400e-b8d6-8bf9462f9ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:45.227702 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:45.282723 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1136/2301] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:21:45.292147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafd43f-5910-4d81-8154-190f95a71eb5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:45.297721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafd43f-5910-4d81-8154-190f95a71eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:45.297721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafd43f-5910-4d81-8154-190f95a71eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:45.313058 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8cafd43f-5910-4d81-8154-190f95a71eb5 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:21:45.314381 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8cafd43f-5910-4d81-8154-190f95a71eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:45.322714 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1166/2302] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:21:45.327766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4d355-53da-400e-b8d6-8bf9462f9ae4 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:45.338341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4d355-53da-400e-b8d6-8bf9462f9ae4 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:45.344515 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea60d3c-4353-4d62-a636-8240fc6b84cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:45.345119 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea60d3c-4353-4d62-a636-8240fc6b84cc admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:45.345399 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea60d3c-4353-4d62-a636-8240fc6b84cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:45.345671 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea60d3c-4353-4d62-a636-8240fc6b84cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:45.346317 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea60d3c-4353-4d62-a636-8240fc6b84cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:21:45.346619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea60d3c-4353-4d62-a636-8240fc6b84cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:45.349308 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea60d3c-4353-4d62-a636-8240fc6b84cc admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:45.361531 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1137/2303] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:21:45.365811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafd43f-5910-4d81-8154-190f95a71eb5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:45.375197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafd43f-5910-4d81-8154-190f95a71eb5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:45.379874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627cdba-6ddd-40d9-803c-63602a988807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:45.380521 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627cdba-6ddd-40d9-803c-63602a988807 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:45.380790 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627cdba-6ddd-40d9-803c-63602a988807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:45.381050 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627cdba-6ddd-40d9-803c-63602a988807 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:45.381695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627cdba-6ddd-40d9-803c-63602a988807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:45.383678 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627cdba-6ddd-40d9-803c-63602a988807 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:45.389747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:45.459850 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1167/2304] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:21:45 2022] POST /identity/v3/projects => generated 381 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:21:45.463884 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea60d3c-4353-4d62-a636-8240fc6b84cc admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:45.474453 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea60d3c-4353-4d62-a636-8240fc6b84cc admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:45.531151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284f3b4-4ab2-4ecc-aa7a-8029f77f6cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:45.531748 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284f3b4-4ab2-4ecc-aa7a-8029f77f6cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:45.532049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284f3b4-4ab2-4ecc-aa7a-8029f77f6cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:45.532322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284f3b4-4ab2-4ecc-aa7a-8029f77f6cf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:45.532957 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284f3b4-4ab2-4ecc-aa7a-8029f77f6cf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:45.534929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284f3b4-4ab2-4ecc-aa7a-8029f77f6cf7 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:45.608207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1138/2305] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:21:45 2022] POST /identity/v3/users => generated 291 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:21:45.612494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627cdba-6ddd-40d9-803c-63602a988807 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:45.622641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627cdba-6ddd-40d9-803c-63602a988807 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:45.627203 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eec32f-8141-46f3-996c-46ddc0b3436a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:45.627816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eec32f-8141-46f3-996c-46ddc0b3436a admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:45.628085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eec32f-8141-46f3-996c-46ddc0b3436a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:45.628345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eec32f-8141-46f3-996c-46ddc0b3436a admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:45.628981 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eec32f-8141-46f3-996c-46ddc0b3436a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:45.630549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eec32f-8141-46f3-996c-46ddc0b3436a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:45.641261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1168/2306] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:21:45.646929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284f3b4-4ab2-4ecc-aa7a-8029f77f6cf7 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:45.657679 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284f3b4-4ab2-4ecc-aa7a-8029f77f6cf7 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:45.662251 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c432ce-f8a2-4bb3-8009-831d50e9aca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:45.662841 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c432ce-f8a2-4bb3-8009-831d50e9aca8 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:45.663150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c432ce-f8a2-4bb3-8009-831d50e9aca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:45.663424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c432ce-f8a2-4bb3-8009-831d50e9aca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:45.664063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c432ce-f8a2-4bb3-8009-831d50e9aca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:45.665638 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c432ce-f8a2-4bb3-8009-831d50e9aca8 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:45.676257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1139/2307] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:21:45.680231 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eec32f-8141-46f3-996c-46ddc0b3436a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:45.690672 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eec32f-8141-46f3-996c-46ddc0b3436a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:45.695280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5307d-faf9-4c8d-9cf1-ec754fbc07df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:45.695911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5307d-faf9-4c8d-9cf1-ec754fbc07df admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:45.696180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5307d-faf9-4c8d-9cf1-ec754fbc07df admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:45.696445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5307d-faf9-4c8d-9cf1-ec754fbc07df admin admin] PATH_INFO: `/v3/projects/67c5047939474c95882cf9e4ec2a8602/users/c7ef2ff567dc49c3b03e4f830cbe9838/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:45.711462 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5307d-faf9-4c8d-9cf1-ec754fbc07df admin admin] RBAC: Authorizing `identity:create_grant(project_id=67c5047939474c95882cf9e4ec2a8602, user_id=c7ef2ff567dc49c3b03e4f830cbe9838, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:45.714254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5307d-faf9-4c8d-9cf1-ec754fbc07df admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:45.795221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1169/2308] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:21:45 2022] PUT /identity/v3/projects/67c5047939474c95882cf9e4ec2a8602/users/c7ef2ff567dc49c3b03e4f830cbe9838/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:45.805296 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8eb8a-1122-4eb3-bd9d-54c23f085e60 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:45.805583 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8eb8a-1122-4eb3-bd9d-54c23f085e60 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:45.805855 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8eb8a-1122-4eb3-bd9d-54c23f085e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:45.834687 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-96b8eb8a-1122-4eb3-bd9d-54c23f085e60 None None] MFA Rules not processed for user `c7ef2ff567dc49c3b03e4f830cbe9838`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:21:45.835927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-96b8eb8a-1122-4eb3-bd9d-54c23f085e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:45.907366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1140/2309] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:21:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:21:45.911859 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5307d-faf9-4c8d-9cf1-ec754fbc07df admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:45.973047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5307d-faf9-4c8d-9cf1-ec754fbc07df admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:45.977646 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a12bc8-d5d7-424d-85ec-d0993eb1a212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:45.978229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a12bc8-d5d7-424d-85ec-d0993eb1a212 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:45.978503 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a12bc8-d5d7-424d-85ec-d0993eb1a212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:45.978764 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a12bc8-d5d7-424d-85ec-d0993eb1a212 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:45.979417 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a12bc8-d5d7-424d-85ec-d0993eb1a212 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:45.981387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a12bc8-d5d7-424d-85ec-d0993eb1a212 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:46.070387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1170/2310] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:21:45 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:21:46.075064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8eb8a-1122-4eb3-bd9d-54c23f085e60 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:46.085825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8eb8a-1122-4eb3-bd9d-54c23f085e60 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:46.090385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:46.141712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06612df7-d150-4319-931f-84d1c7032da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:46.142361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06612df7-d150-4319-931f-84d1c7032da5 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:46.142841 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06612df7-d150-4319-931f-84d1c7032da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:46.143352 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06612df7-d150-4319-931f-84d1c7032da5 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:46.144039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06612df7-d150-4319-931f-84d1c7032da5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:46.146517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06612df7-d150-4319-931f-84d1c7032da5 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:46.237045 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1141/2311] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:21:46 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:21:46.241031 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a12bc8-d5d7-424d-85ec-d0993eb1a212 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:46.251964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a12bc8-d5d7-424d-85ec-d0993eb1a212 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:46.256696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5c249-0fae-4f62-8e9e-8e342109e2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:46.257282 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5c249-0fae-4f62-8e9e-8e342109e2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:46.257558 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5c249-0fae-4f62-8e9e-8e342109e2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:46.257825 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5c249-0fae-4f62-8e9e-8e342109e2d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:46.258463 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5c249-0fae-4f62-8e9e-8e342109e2d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:46.260087 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5c249-0fae-4f62-8e9e-8e342109e2d7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:46.270207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1171/2312] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:21:46.274279 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06612df7-d150-4319-931f-84d1c7032da5 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:46.284310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06612df7-d150-4319-931f-84d1c7032da5 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:46.288994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cb99e-211d-4a83-99b1-9a48aee8b384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:46.289634 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cb99e-211d-4a83-99b1-9a48aee8b384 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:46.289918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cb99e-211d-4a83-99b1-9a48aee8b384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:46.290190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cb99e-211d-4a83-99b1-9a48aee8b384 admin admin] PATH_INFO: `/v3/projects/00dd6ca2ba3c443889c3548401d21179/users/c8e0e625c396454bb9a44182e584bde4/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:46.306042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cb99e-211d-4a83-99b1-9a48aee8b384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00dd6ca2ba3c443889c3548401d21179, user_id=c8e0e625c396454bb9a44182e584bde4, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:46.309514 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cb99e-211d-4a83-99b1-9a48aee8b384 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:46.441184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1142/2313] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:21:46 2022] PUT /identity/v3/projects/00dd6ca2ba3c443889c3548401d21179/users/c8e0e625c396454bb9a44182e584bde4/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:46.445278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5c249-0fae-4f62-8e9e-8e342109e2d7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:46.455208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5c249-0fae-4f62-8e9e-8e342109e2d7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:46.511303 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28def4dd-9533-4118-b147-96ed0bce46a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:46.512266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28def4dd-9533-4118-b147-96ed0bce46a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:46.512539 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28def4dd-9533-4118-b147-96ed0bce46a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:46.512810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28def4dd-9533-4118-b147-96ed0bce46a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:46.513441 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28def4dd-9533-4118-b147-96ed0bce46a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:46.515050 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28def4dd-9533-4118-b147-96ed0bce46a7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:46.525057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1172/2314] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:21:46.529852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cb99e-211d-4a83-99b1-9a48aee8b384 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:46.540221 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cb99e-211d-4a83-99b1-9a48aee8b384 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:46.545654 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f08b97-9383-4778-a3f3-7c37cd4e4ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:46.546287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f08b97-9383-4778-a3f3-7c37cd4e4ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:46.546558 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f08b97-9383-4778-a3f3-7c37cd4e4ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:46.546824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f08b97-9383-4778-a3f3-7c37cd4e4ed8 admin admin] PATH_INFO: `/v3/projects/00dd6ca2ba3c443889c3548401d21179/users/c8e0e625c396454bb9a44182e584bde4/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:46.548704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f08b97-9383-4778-a3f3-7c37cd4e4ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00dd6ca2ba3c443889c3548401d21179, user_id=c8e0e625c396454bb9a44182e584bde4, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:46.551353 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f08b97-9383-4778-a3f3-7c37cd4e4ed8 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:46.561536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1143/2315] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:21:46 2022] PUT /identity/v3/projects/00dd6ca2ba3c443889c3548401d21179/users/c8e0e625c396454bb9a44182e584bde4/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:46.571561 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066aefd-ef31-4c7c-8653-644d6999bf52 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:46.571843 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066aefd-ef31-4c7c-8653-644d6999bf52 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:46.572115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066aefd-ef31-4c7c-8653-644d6999bf52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:46.600106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d066aefd-ef31-4c7c-8653-644d6999bf52 None None] MFA Rules not processed for user `c8e0e625c396454bb9a44182e584bde4`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:21:46.601289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d066aefd-ef31-4c7c-8653-644d6999bf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:46.610805 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:46.668691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1173/2316] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:21:46 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:21:46.679543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f08b97-9383-4778-a3f3-7c37cd4e4ed8 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:46.681499 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e0f08b97-9383-4778-a3f3-7c37cd4e4ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:46.737495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f08b97-9383-4778-a3f3-7c37cd4e4ed8 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:46.741132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46d2d-c5dd-4a99-bcb8-ae8d4fccb5cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:46.741719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46d2d-c5dd-4a99-bcb8-ae8d4fccb5cf service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:46.741993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46d2d-c5dd-4a99-bcb8-ae8d4fccb5cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:46.742284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46d2d-c5dd-4a99-bcb8-ae8d4fccb5cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:46.752047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb46d2d-c5dd-4a99-bcb8-ae8d4fccb5cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:46.753476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb46d2d-c5dd-4a99-bcb8-ae8d4fccb5cf service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:46.763096 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1144/2317] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:21:46 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:21:46.874252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066aefd-ef31-4c7c-8653-644d6999bf52 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:46.883539 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066aefd-ef31-4c7c-8653-644d6999bf52 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:46.887961 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c47f9-1eb9-4d38-88e4-74c84156ba8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:46.888544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c47f9-1eb9-4d38-88e4-74c84156ba8d service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:46.888814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c47f9-1eb9-4d38-88e4-74c84156ba8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:46.889077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c47f9-1eb9-4d38-88e4-74c84156ba8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:46.944773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c47f9-1eb9-4d38-88e4-74c84156ba8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:46.946188 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c47f9-1eb9-4d38-88e4-74c84156ba8d service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:46.954876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1174/2318] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:21:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:21:51.211302 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46d2d-c5dd-4a99-bcb8-ae8d4fccb5cf service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:51.272600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46d2d-c5dd-4a99-bcb8-ae8d4fccb5cf service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:51.277191 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f9fbb-2a1d-4f6b-88e4-3a7049bf6850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:51.277806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f9fbb-2a1d-4f6b-88e4-3a7049bf6850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:51.278079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f9fbb-2a1d-4f6b-88e4-3a7049bf6850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:51.278343 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f9fbb-2a1d-4f6b-88e4-3a7049bf6850 admin admin] PATH_INFO: `/v3/users/c7ef2ff567dc49c3b03e4f830cbe9838` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:51.279492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f9fbb-2a1d-4f6b-88e4-3a7049bf6850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7ef2ff567dc49c3b03e4f830cbe9838)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:51.281393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f9fbb-2a1d-4f6b-88e4-3a7049bf6850 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:51.285889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:51.444319 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-084f9fbb-2a1d-4f6b-88e4-3a7049bf6850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7ef2ff567dc49c3b03e4f830cbe9838'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:51.444893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-084f9fbb-2a1d-4f6b-88e4-3a7049bf6850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7ef2ff567dc49c3b03e4f830cbe9838'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:51.456871 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-084f9fbb-2a1d-4f6b-88e4-3a7049bf6850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7ef2ff567dc49c3b03e4f830cbe9838'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:51.510262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-084f9fbb-2a1d-4f6b-88e4-3a7049bf6850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7ef2ff567dc49c3b03e4f830cbe9838'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:51.511346 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-084f9fbb-2a1d-4f6b-88e4-3a7049bf6850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7ef2ff567dc49c3b03e4f830cbe9838'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:51.524140 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1145/2319] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:21:51 2022] DELETE /identity/v3/users/c7ef2ff567dc49c3b03e4f830cbe9838 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:51.529157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a71d14-7c56-483d-83a1-57181956add2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:51.529668 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a71d14-7c56-483d-83a1-57181956add2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:51.529785 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a71d14-7c56-483d-83a1-57181956add2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:51.549753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e2a71d14-7c56-483d-83a1-57181956add2 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:21:51.550969 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e2a71d14-7c56-483d-83a1-57181956add2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:51.609071 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1175/2320] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:21:51.618806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f9fbb-2a1d-4f6b-88e4-3a7049bf6850 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:51.620879 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-084f9fbb-2a1d-4f6b-88e4-3a7049bf6850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:51.677528 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f9fbb-2a1d-4f6b-88e4-3a7049bf6850 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:51.681151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e24cd-7dce-4aa8-a31d-05f14ec3bec7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:51.681739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e24cd-7dce-4aa8-a31d-05f14ec3bec7 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:51.682010 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e24cd-7dce-4aa8-a31d-05f14ec3bec7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:51.682276 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e24cd-7dce-4aa8-a31d-05f14ec3bec7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:51.692061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e24cd-7dce-4aa8-a31d-05f14ec3bec7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:51.693479 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e24cd-7dce-4aa8-a31d-05f14ec3bec7 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:51.703151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1146/2321] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:21:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:21:52.087494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a71d14-7c56-483d-83a1-57181956add2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:52.088957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e2a71d14-7c56-483d-83a1-57181956add2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:52.102823 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a71d14-7c56-483d-83a1-57181956add2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:52.111179 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:52.111179 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:52.111179 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:52.111179 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75 admin admin] PATH_INFO: `/v3/projects/67c5047939474c95882cf9e4ec2a8602` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:52.111823 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67c5047939474c95882cf9e4ec2a8602)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:52.112815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:52.230452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:52.236489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67c5047939474c95882cf9e4ec2a8602', 'request_id': 'req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:52.242587 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67c5047939474c95882cf9e4ec2a8602', 'request_id': 'req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:52.326435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67c5047939474c95882cf9e4ec2a8602', 'request_id': 'req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:52.337401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75 admin admin] The token cache is being invalidate because project 67c5047939474c95882cf9e4ec2a8602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:21:52.337692 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:52.338605 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:52.341514 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1176/2322] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:21:52 2022] DELETE /identity/v3/projects/67c5047939474c95882cf9e4ec2a8602 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:52.345631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e24cd-7dce-4aa8-a31d-05f14ec3bec7 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:52.356020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e24cd-7dce-4aa8-a31d-05f14ec3bec7 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:52.413243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee942ca-b180-435d-9a13-571fe582cd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:52.416873 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee942ca-b180-435d-9a13-571fe582cd8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:52.416873 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee942ca-b180-435d-9a13-571fe582cd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:52.416873 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee942ca-b180-435d-9a13-571fe582cd8d admin admin] PATH_INFO: `/v3/users/c8e0e625c396454bb9a44182e584bde4` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:52.416873 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee942ca-b180-435d-9a13-571fe582cd8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8e0e625c396454bb9a44182e584bde4)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:52.418042 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee942ca-b180-435d-9a13-571fe582cd8d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:52.500085 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:52.615259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-5ee942ca-b180-435d-9a13-571fe582cd8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8e0e625c396454bb9a44182e584bde4'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:52.615918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-5ee942ca-b180-435d-9a13-571fe582cd8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8e0e625c396454bb9a44182e584bde4'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:52.628403 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-5ee942ca-b180-435d-9a13-571fe582cd8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8e0e625c396454bb9a44182e584bde4'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:52.737936 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-5ee942ca-b180-435d-9a13-571fe582cd8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8e0e625c396454bb9a44182e584bde4'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:52.738590 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-5ee942ca-b180-435d-9a13-571fe582cd8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8e0e625c396454bb9a44182e584bde4'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:52.752098 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1147/2323] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:21:52 2022] DELETE /identity/v3/users/c8e0e625c396454bb9a44182e584bde4 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:53.163217 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:53.164830 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:53.178654 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee2e57-62b5-4c21-9a1f-695f23e1dc75 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:53.243259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9262579-f37d-4dd0-bf75-153bb0bc00ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:53.243892 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9262579-f37d-4dd0-bf75-153bb0bc00ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:53.244174 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9262579-f37d-4dd0-bf75-153bb0bc00ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:53.244473 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9262579-f37d-4dd0-bf75-153bb0bc00ed admin admin] PATH_INFO: `/v3/projects/00dd6ca2ba3c443889c3548401d21179` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:53.246160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9262579-f37d-4dd0-bf75-153bb0bc00ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=00dd6ca2ba3c443889c3548401d21179)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:53.248163 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9262579-f37d-4dd0-bf75-153bb0bc00ed admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:53.426992 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d9262579-f37d-4dd0-bf75-153bb0bc00ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00dd6ca2ba3c443889c3548401d21179', 'request_id': 'req-d9262579-f37d-4dd0-bf75-153bb0bc00ed'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:53.434154 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d9262579-f37d-4dd0-bf75-153bb0bc00ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00dd6ca2ba3c443889c3548401d21179', 'request_id': 'req-d9262579-f37d-4dd0-bf75-153bb0bc00ed'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:53.527419 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d9262579-f37d-4dd0-bf75-153bb0bc00ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00dd6ca2ba3c443889c3548401d21179', 'request_id': 'req-d9262579-f37d-4dd0-bf75-153bb0bc00ed'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:53.538445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d9262579-f37d-4dd0-bf75-153bb0bc00ed admin admin] The token cache is being invalidate because project 00dd6ca2ba3c443889c3548401d21179 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:21:53.538727 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d9262579-f37d-4dd0-bf75-153bb0bc00ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:53.539388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d9262579-f37d-4dd0-bf75-153bb0bc00ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:21:53.540866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1177/2324] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:21:53 2022] DELETE /identity/v3/projects/00dd6ca2ba3c443889c3548401d21179 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:53.546496 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249059b4-ec50-4f9f-8c31-c945f43eb1a3 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:53.547018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249059b4-ec50-4f9f-8c31-c945f43eb1a3 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:53.547114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249059b4-ec50-4f9f-8c31-c945f43eb1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:53.566918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-249059b4-ec50-4f9f-8c31-c945f43eb1a3 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:21:53.568183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-249059b4-ec50-4f9f-8c31-c945f43eb1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:53.628826 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1148/2325] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:21:53.638153 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d39b0-fd2b-450c-9bb7-53cf3e34acde None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:53.638511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d39b0-fd2b-450c-9bb7-53cf3e34acde None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:53.638975 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d39b0-fd2b-450c-9bb7-53cf3e34acde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:53.659040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5f9d39b0-fd2b-450c-9bb7-53cf3e34acde None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:21:53.660246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5f9d39b0-fd2b-450c-9bb7-53cf3e34acde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:53.668913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1178/2326] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:21:53.673189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249059b4-ec50-4f9f-8c31-c945f43eb1a3 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:53.682742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249059b4-ec50-4f9f-8c31-c945f43eb1a3 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:53.688626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cebc50-51ad-43a6-830c-90408ae48b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:53.689222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cebc50-51ad-43a6-830c-90408ae48b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:53.689498 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cebc50-51ad-43a6-830c-90408ae48b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:53.690607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cebc50-51ad-43a6-830c-90408ae48b05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:53.690607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cebc50-51ad-43a6-830c-90408ae48b05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:21:53.690838 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cebc50-51ad-43a6-830c-90408ae48b05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:53.692451 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cebc50-51ad-43a6-830c-90408ae48b05 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:53.704654 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1149/2327] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:21:53.708913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d39b0-fd2b-450c-9bb7-53cf3e34acde None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:53.718516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d39b0-fd2b-450c-9bb7-53cf3e34acde None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:53.723363 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6fb83-5abc-41cb-a135-b153b8ab98c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:53.723966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6fb83-5abc-41cb-a135-b153b8ab98c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:53.724241 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6fb83-5abc-41cb-a135-b153b8ab98c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:53.724507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6fb83-5abc-41cb-a135-b153b8ab98c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:53.725150 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6fb83-5abc-41cb-a135-b153b8ab98c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:53.727101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6fb83-5abc-41cb-a135-b153b8ab98c4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:53.861610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1179/2328] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:21:53 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:21:53.865982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cebc50-51ad-43a6-830c-90408ae48b05 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:53.877024 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cebc50-51ad-43a6-830c-90408ae48b05 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:53.937920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb9302-b8b9-4cd8-b741-91599cce6f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:53.938533 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb9302-b8b9-4cd8-b741-91599cce6f61 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:53.938811 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb9302-b8b9-4cd8-b741-91599cce6f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:53.939142 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb9302-b8b9-4cd8-b741-91599cce6f61 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:53.939818 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb9302-b8b9-4cd8-b741-91599cce6f61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:53.941822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb9302-b8b9-4cd8-b741-91599cce6f61 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:54.058858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1150/2329] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:21:53 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:21:54.063195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6fb83-5abc-41cb-a135-b153b8ab98c4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:54.073222 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6fb83-5abc-41cb-a135-b153b8ab98c4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:54.078004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e94b8c-28ae-4fcd-9cff-027674febe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:54.085386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e94b8c-28ae-4fcd-9cff-027674febe32 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:54.085386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e94b8c-28ae-4fcd-9cff-027674febe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:54.085386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e94b8c-28ae-4fcd-9cff-027674febe32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:54.085386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e94b8c-28ae-4fcd-9cff-027674febe32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:54.085386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e94b8c-28ae-4fcd-9cff-027674febe32 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:54.092551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1180/2330] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:21:54.095765 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb9302-b8b9-4cd8-b741-91599cce6f61 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:54.105692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb9302-b8b9-4cd8-b741-91599cce6f61 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:54.111521 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1a8fe-fd9c-4b22-9ae0-913e3305af65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:54.112102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1a8fe-fd9c-4b22-9ae0-913e3305af65 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:54.112377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1a8fe-fd9c-4b22-9ae0-913e3305af65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:54.112643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1a8fe-fd9c-4b22-9ae0-913e3305af65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:54.113280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1a8fe-fd9c-4b22-9ae0-913e3305af65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:54.114901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1a8fe-fd9c-4b22-9ae0-913e3305af65 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:54.126411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1151/2331] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:21:54.130715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e94b8c-28ae-4fcd-9cff-027674febe32 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:54.141222 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e94b8c-28ae-4fcd-9cff-027674febe32 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:54.146728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd86954-a881-4c9c-a244-be8f87bf7b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:54.147424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd86954-a881-4c9c-a244-be8f87bf7b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:54.147704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd86954-a881-4c9c-a244-be8f87bf7b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:54.147973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd86954-a881-4c9c-a244-be8f87bf7b9b admin admin] PATH_INFO: `/v3/projects/367cb4d92ecc45d2ab498081f62f1335/users/f0a952ca380c4845ab0e17e2f5a3266e/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:54.163570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd86954-a881-4c9c-a244-be8f87bf7b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=367cb4d92ecc45d2ab498081f62f1335, user_id=f0a952ca380c4845ab0e17e2f5a3266e, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:54.166212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd86954-a881-4c9c-a244-be8f87bf7b9b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:54.169903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:54.242140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1181/2332] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:21:54 2022] PUT /identity/v3/projects/367cb4d92ecc45d2ab498081f62f1335/users/f0a952ca380c4845ab0e17e2f5a3266e/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:54.252472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937d0d3-5a61-4c04-95ed-7918f46e605e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:54.252750 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937d0d3-5a61-4c04-95ed-7918f46e605e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:54.253021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937d0d3-5a61-4c04-95ed-7918f46e605e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:54.275481 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:54.284000 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-a937d0d3-5a61-4c04-95ed-7918f46e605e None None] MFA Rules not processed for user `f0a952ca380c4845ab0e17e2f5a3266e`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:21:54.285240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a937d0d3-5a61-4c04-95ed-7918f46e605e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:54.353297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1152/2333] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:21:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:21:54.358006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd86954-a881-4c9c-a244-be8f87bf7b9b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:54.421000 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd86954-a881-4c9c-a244-be8f87bf7b9b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:54.425754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2289005-c75d-4a80-bc04-3cc8addb5226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:54.426346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2289005-c75d-4a80-bc04-3cc8addb5226 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:54.426621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2289005-c75d-4a80-bc04-3cc8addb5226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:54.426891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2289005-c75d-4a80-bc04-3cc8addb5226 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:54.427552 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2289005-c75d-4a80-bc04-3cc8addb5226 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:54.429503 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2289005-c75d-4a80-bc04-3cc8addb5226 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:54.517314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1182/2334] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:21:54 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:21:54.521531 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a937d0d3-5a61-4c04-95ed-7918f46e605e None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:54.533414 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a937d0d3-5a61-4c04-95ed-7918f46e605e None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:54.588961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d2101-4e18-4b44-be4a-33da196dd93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:54.589597 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d2101-4e18-4b44-be4a-33da196dd93c admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:54.589820 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d2101-4e18-4b44-be4a-33da196dd93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:54.590085 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d2101-4e18-4b44-be4a-33da196dd93c admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:54.590701 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d2101-4e18-4b44-be4a-33da196dd93c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:54.593263 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d2101-4e18-4b44-be4a-33da196dd93c admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:54.681380 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1153/2335] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:21:54 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:21:54.686219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2289005-c75d-4a80-bc04-3cc8addb5226 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:54.696478 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2289005-c75d-4a80-bc04-3cc8addb5226 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:54.702410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd31e8-8819-4f75-932b-837978386b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:54.703197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd31e8-8819-4f75-932b-837978386b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:54.703430 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd31e8-8819-4f75-932b-837978386b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:54.703696 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd31e8-8819-4f75-932b-837978386b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:54.704337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd31e8-8819-4f75-932b-837978386b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:54.705954 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd31e8-8819-4f75-932b-837978386b05 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:54.717072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1183/2336] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:21:54.721145 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d2101-4e18-4b44-be4a-33da196dd93c admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:54.731016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d2101-4e18-4b44-be4a-33da196dd93c admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:54.735656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073eeec1-1e63-4edc-a58e-62a200501d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:54.736291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073eeec1-1e63-4edc-a58e-62a200501d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:54.736568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073eeec1-1e63-4edc-a58e-62a200501d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:54.736837 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073eeec1-1e63-4edc-a58e-62a200501d8e admin admin] PATH_INFO: `/v3/projects/ba90bed47058488f8ff35a185661b039/users/89b782e5a3ff493ab3b6a8ff912a3ce3/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:54.751649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073eeec1-1e63-4edc-a58e-62a200501d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba90bed47058488f8ff35a185661b039, user_id=89b782e5a3ff493ab3b6a8ff912a3ce3, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:54.754298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073eeec1-1e63-4edc-a58e-62a200501d8e admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:54.825021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1154/2337] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:21:54 2022] PUT /identity/v3/projects/ba90bed47058488f8ff35a185661b039/users/89b782e5a3ff493ab3b6a8ff912a3ce3/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:54.829814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd31e8-8819-4f75-932b-837978386b05 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:54.839610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd31e8-8819-4f75-932b-837978386b05 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:54.894810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0138c-e0a6-4fe2-99f0-fd5b7802bfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:54.895430 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0138c-e0a6-4fe2-99f0-fd5b7802bfcf admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:54.895702 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0138c-e0a6-4fe2-99f0-fd5b7802bfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:54.895964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0138c-e0a6-4fe2-99f0-fd5b7802bfcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:54.896589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0138c-e0a6-4fe2-99f0-fd5b7802bfcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:54.898156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0138c-e0a6-4fe2-99f0-fd5b7802bfcf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:54.907922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1184/2338] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:21:54.912094 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073eeec1-1e63-4edc-a58e-62a200501d8e admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:54.922523 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073eeec1-1e63-4edc-a58e-62a200501d8e admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:54.927636 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2125017-227a-4336-bf19-8e8799937397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:54.928128 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2125017-227a-4336-bf19-8e8799937397 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:54.928419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2125017-227a-4336-bf19-8e8799937397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:54.928694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2125017-227a-4336-bf19-8e8799937397 admin admin] PATH_INFO: `/v3/projects/ba90bed47058488f8ff35a185661b039/users/89b782e5a3ff493ab3b6a8ff912a3ce3/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:54.930609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2125017-227a-4336-bf19-8e8799937397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba90bed47058488f8ff35a185661b039, user_id=89b782e5a3ff493ab3b6a8ff912a3ce3, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:54.933513 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2125017-227a-4336-bf19-8e8799937397 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:54.944453 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1155/2339] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:21:54 2022] PUT /identity/v3/projects/ba90bed47058488f8ff35a185661b039/users/89b782e5a3ff493ab3b6a8ff912a3ce3/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:21:54.955181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70923c-312b-4222-af9a-b2f8b58a7046 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:54.955460 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70923c-312b-4222-af9a-b2f8b58a7046 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:54.955746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70923c-312b-4222-af9a-b2f8b58a7046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:54.982694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5c70923c-312b-4222-af9a-b2f8b58a7046 None None] MFA Rules not processed for user `89b782e5a3ff493ab3b6a8ff912a3ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:21:54.983915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5c70923c-312b-4222-af9a-b2f8b58a7046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:55.033614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:55.051297 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1185/2340] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:21:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:21:55.062630 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2125017-227a-4336-bf19-8e8799937397 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:21:55.064321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a2125017-227a-4336-bf19-8e8799937397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:21:55.121925 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2125017-227a-4336-bf19-8e8799937397 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:21:55.126262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298454c-aa1e-489d-afa1-84d4040a083a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:21:55.126854 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298454c-aa1e-489d-afa1-84d4040a083a service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:21:55.127152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298454c-aa1e-489d-afa1-84d4040a083a service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:21:55.127418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298454c-aa1e-489d-afa1-84d4040a083a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:21:55.132546 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:21:55.183932 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298454c-aa1e-489d-afa1-84d4040a083a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:21:55.185944 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298454c-aa1e-489d-afa1-84d4040a083a service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:21:55.194816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1156/2341] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:21:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:22:00.847791 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70923c-312b-4222-af9a-b2f8b58a7046 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:00.857338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70923c-312b-4222-af9a-b2f8b58a7046 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:00.861035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99f6aa-ee1a-49c7-b91b-01ab78f73998 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:00.861598 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99f6aa-ee1a-49c7-b91b-01ab78f73998 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:00.861917 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99f6aa-ee1a-49c7-b91b-01ab78f73998 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:00.862183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99f6aa-ee1a-49c7-b91b-01ab78f73998 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:00.871614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99f6aa-ee1a-49c7-b91b-01ab78f73998 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:00.873686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99f6aa-ee1a-49c7-b91b-01ab78f73998 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:00.883304 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1186/2342] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:22:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:22:05.235894 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=24 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:22:07.887514 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298454c-aa1e-489d-afa1-84d4040a083a service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:07.954449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298454c-aa1e-489d-afa1-84d4040a083a service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:07.959156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e058cb0-92cf-4ed2-ad10-255784bec38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:07.959784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e058cb0-92cf-4ed2-ad10-255784bec38d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:07.960089 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e058cb0-92cf-4ed2-ad10-255784bec38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:07.960357 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e058cb0-92cf-4ed2-ad10-255784bec38d admin admin] PATH_INFO: `/v3/users/f0a952ca380c4845ab0e17e2f5a3266e` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:07.961473 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e058cb0-92cf-4ed2-ad10-255784bec38d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0a952ca380c4845ab0e17e2f5a3266e)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:07.963459 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e058cb0-92cf-4ed2-ad10-255784bec38d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:08.129084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6e058cb0-92cf-4ed2-ad10-255784bec38d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0a952ca380c4845ab0e17e2f5a3266e'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:08.129670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6e058cb0-92cf-4ed2-ad10-255784bec38d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0a952ca380c4845ab0e17e2f5a3266e'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:08.142729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6e058cb0-92cf-4ed2-ad10-255784bec38d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0a952ca380c4845ab0e17e2f5a3266e'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:08.240423 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6e058cb0-92cf-4ed2-ad10-255784bec38d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0a952ca380c4845ab0e17e2f5a3266e'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:08.241072 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6e058cb0-92cf-4ed2-ad10-255784bec38d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0a952ca380c4845ab0e17e2f5a3266e'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:08.254384 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1157/2343] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:22:07 2022] DELETE /identity/v3/users/f0a952ca380c4845ab0e17e2f5a3266e => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:22:08.259122 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd86ba3-9f31-498e-a8b9-a21fc2d7f432 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:08.259556 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd86ba3-9f31-498e-a8b9-a21fc2d7f432 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:08.259661 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd86ba3-9f31-498e-a8b9-a21fc2d7f432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:08.279608 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1cd86ba3-9f31-498e-a8b9-a21fc2d7f432 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:08.281117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1cd86ba3-9f31-498e-a8b9-a21fc2d7f432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:08.340136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1187/2344] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:22:08.349869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e058cb0-92cf-4ed2-ad10-255784bec38d admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:08.351328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6e058cb0-92cf-4ed2-ad10-255784bec38d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:08.407455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e058cb0-92cf-4ed2-ad10-255784bec38d admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:08.410934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bf330-db65-4553-8adb-5b3ba78c16ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:08.411549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bf330-db65-4553-8adb-5b3ba78c16ff service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:08.411821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bf330-db65-4553-8adb-5b3ba78c16ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:08.412084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bf330-db65-4553-8adb-5b3ba78c16ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:08.421020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bf330-db65-4553-8adb-5b3ba78c16ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:08.422441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bf330-db65-4553-8adb-5b3ba78c16ff service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:08.431554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1158/2345] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:22:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:22:08.824844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd86ba3-9f31-498e-a8b9-a21fc2d7f432 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:08.826480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1cd86ba3-9f31-498e-a8b9-a21fc2d7f432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:08.838328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd86ba3-9f31-498e-a8b9-a21fc2d7f432 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:08.844147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:08.844782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:08.845190 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:08.845479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18 admin admin] PATH_INFO: `/v3/projects/367cb4d92ecc45d2ab498081f62f1335` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:08.846605 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=367cb4d92ecc45d2ab498081f62f1335)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:08.848558 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:08.955977 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '367cb4d92ecc45d2ab498081f62f1335', 'request_id': 'req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:08.962029 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '367cb4d92ecc45d2ab498081f62f1335', 'request_id': 'req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:09.042026 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '367cb4d92ecc45d2ab498081f62f1335', 'request_id': 'req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:09.055716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18 admin admin] The token cache is being invalidate because project 367cb4d92ecc45d2ab498081f62f1335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:22:09.056007 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:09.056660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:09.058194 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1188/2346] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:22:08 2022] DELETE /identity/v3/projects/367cb4d92ecc45d2ab498081f62f1335 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:22:09.062268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bf330-db65-4553-8adb-5b3ba78c16ff service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:09.072821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bf330-db65-4553-8adb-5b3ba78c16ff service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:09.091417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:22:09.133759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d1a51-4370-4adf-85a3-76b6b52b2081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:09.134893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d1a51-4370-4adf-85a3-76b6b52b2081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:09.135173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d1a51-4370-4adf-85a3-76b6b52b2081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:09.135513 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d1a51-4370-4adf-85a3-76b6b52b2081 admin admin] PATH_INFO: `/v3/users/89b782e5a3ff493ab3b6a8ff912a3ce3` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:09.136647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d1a51-4370-4adf-85a3-76b6b52b2081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89b782e5a3ff493ab3b6a8ff912a3ce3)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:09.138923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d1a51-4370-4adf-85a3-76b6b52b2081 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:09.330815 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3f1d1a51-4370-4adf-85a3-76b6b52b2081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89b782e5a3ff493ab3b6a8ff912a3ce3'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:09.331510 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3f1d1a51-4370-4adf-85a3-76b6b52b2081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89b782e5a3ff493ab3b6a8ff912a3ce3'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:09.344785 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3f1d1a51-4370-4adf-85a3-76b6b52b2081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89b782e5a3ff493ab3b6a8ff912a3ce3'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:09.418786 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3f1d1a51-4370-4adf-85a3-76b6b52b2081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89b782e5a3ff493ab3b6a8ff912a3ce3'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:09.419446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3f1d1a51-4370-4adf-85a3-76b6b52b2081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89b782e5a3ff493ab3b6a8ff912a3ce3'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:09.433114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1159/2347] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:22:09 2022] DELETE /identity/v3/users/89b782e5a3ff493ab3b6a8ff912a3ce3 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:22:09.832518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:09.834042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:09.846826 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34f3bb-ee0a-425e-9b60-280bd1aeee18 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:09.906340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9006838-87c3-42f9-8a3d-31bae41f7cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:09.907000 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9006838-87c3-42f9-8a3d-31bae41f7cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:09.907275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9006838-87c3-42f9-8a3d-31bae41f7cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:09.907542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9006838-87c3-42f9-8a3d-31bae41f7cde admin admin] PATH_INFO: `/v3/projects/ba90bed47058488f8ff35a185661b039` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:09.908644 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9006838-87c3-42f9-8a3d-31bae41f7cde admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba90bed47058488f8ff35a185661b039)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:09.910620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9006838-87c3-42f9-8a3d-31bae41f7cde admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:09.913227 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:22:10.019064 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b9006838-87c3-42f9-8a3d-31bae41f7cde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba90bed47058488f8ff35a185661b039', 'request_id': 'req-b9006838-87c3-42f9-8a3d-31bae41f7cde'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:10.026854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b9006838-87c3-42f9-8a3d-31bae41f7cde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba90bed47058488f8ff35a185661b039', 'request_id': 'req-b9006838-87c3-42f9-8a3d-31bae41f7cde'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:10.098742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b9006838-87c3-42f9-8a3d-31bae41f7cde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba90bed47058488f8ff35a185661b039', 'request_id': 'req-b9006838-87c3-42f9-8a3d-31bae41f7cde'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:10.110072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b9006838-87c3-42f9-8a3d-31bae41f7cde admin admin] The token cache is being invalidate because project ba90bed47058488f8ff35a185661b039 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:22:10.110362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b9006838-87c3-42f9-8a3d-31bae41f7cde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:10.111092 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b9006838-87c3-42f9-8a3d-31bae41f7cde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:10.112632 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1189/2348] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:22:09 2022] DELETE /identity/v3/projects/ba90bed47058488f8ff35a185661b039 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:22:10.118507 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf8672-197e-466a-b0ef-81c49b521c79 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:10.119035 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf8672-197e-466a-b0ef-81c49b521c79 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:10.119133 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf8672-197e-466a-b0ef-81c49b521c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:10.139957 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-dbdf8672-197e-466a-b0ef-81c49b521c79 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:10.141215 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-dbdf8672-197e-466a-b0ef-81c49b521c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:10.204062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1160/2349] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:22:10.215486 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23595eb3-f830-4c3d-a751-bf84deef5a14 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:10.215486 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23595eb3-f830-4c3d-a751-bf84deef5a14 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:10.215486 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23595eb3-f830-4c3d-a751-bf84deef5a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:10.235305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-23595eb3-f830-4c3d-a751-bf84deef5a14 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:10.236560 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-23595eb3-f830-4c3d-a751-bf84deef5a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:10.244457 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1190/2350] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:22:10.248831 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf8672-197e-466a-b0ef-81c49b521c79 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:10.258845 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf8672-197e-466a-b0ef-81c49b521c79 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:10.263905 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa574a4-f100-4f0f-8d6f-24085f6c32ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:10.264645 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa574a4-f100-4f0f-8d6f-24085f6c32ae admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:10.264929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa574a4-f100-4f0f-8d6f-24085f6c32ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:10.265226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa574a4-f100-4f0f-8d6f-24085f6c32ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:10.265907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa574a4-f100-4f0f-8d6f-24085f6c32ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:22:10.266185 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa574a4-f100-4f0f-8d6f-24085f6c32ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:10.267807 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa574a4-f100-4f0f-8d6f-24085f6c32ae admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:10.280226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1161/2351] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:22:10.284630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23595eb3-f830-4c3d-a751-bf84deef5a14 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:10.294047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23595eb3-f830-4c3d-a751-bf84deef5a14 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:10.299479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df68c1-e7b8-4bb2-bb00-2fa040b85141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:10.300076 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df68c1-e7b8-4bb2-bb00-2fa040b85141 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:10.300349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df68c1-e7b8-4bb2-bb00-2fa040b85141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:10.300614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df68c1-e7b8-4bb2-bb00-2fa040b85141 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:10.301236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df68c1-e7b8-4bb2-bb00-2fa040b85141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:10.303290 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df68c1-e7b8-4bb2-bb00-2fa040b85141 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:10.420410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1191/2352] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:22:10 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:22:10.427116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa574a4-f100-4f0f-8d6f-24085f6c32ae admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:10.440681 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa574a4-f100-4f0f-8d6f-24085f6c32ae admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:10.466631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:22:10.495792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a345c4-9e0c-430f-a6c8-76b3c7c0ed11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:10.496396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a345c4-9e0c-430f-a6c8-76b3c7c0ed11 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:10.496673 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a345c4-9e0c-430f-a6c8-76b3c7c0ed11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:10.496944 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a345c4-9e0c-430f-a6c8-76b3c7c0ed11 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:10.497582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a345c4-9e0c-430f-a6c8-76b3c7c0ed11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:10.500094 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a345c4-9e0c-430f-a6c8-76b3c7c0ed11 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:10.589643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1162/2353] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:22:10 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:22:10.593695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df68c1-e7b8-4bb2-bb00-2fa040b85141 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:10.604396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df68c1-e7b8-4bb2-bb00-2fa040b85141 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:10.608997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9984d9-8363-4c7c-8948-29d6617c88ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:10.609574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9984d9-8363-4c7c-8948-29d6617c88ca admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:10.609851 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9984d9-8363-4c7c-8948-29d6617c88ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:10.610116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9984d9-8363-4c7c-8948-29d6617c88ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:10.610747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9984d9-8363-4c7c-8948-29d6617c88ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:10.612369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9984d9-8363-4c7c-8948-29d6617c88ca admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:10.622108 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1192/2354] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:22:10.627092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a345c4-9e0c-430f-a6c8-76b3c7c0ed11 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:10.636959 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a345c4-9e0c-430f-a6c8-76b3c7c0ed11 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:10.642961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b289ad8-1f0e-4095-9d66-ae7d1ac42517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:10.643547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b289ad8-1f0e-4095-9d66-ae7d1ac42517 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:10.643823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b289ad8-1f0e-4095-9d66-ae7d1ac42517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:10.644117 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b289ad8-1f0e-4095-9d66-ae7d1ac42517 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:10.644753 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b289ad8-1f0e-4095-9d66-ae7d1ac42517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:10.646337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b289ad8-1f0e-4095-9d66-ae7d1ac42517 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:10.657233 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1163/2355] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:22:10.661242 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9984d9-8363-4c7c-8948-29d6617c88ca admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:10.670966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9984d9-8363-4c7c-8948-29d6617c88ca admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:10.676933 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d20ab-d23e-41e3-a933-d8eac975fdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:10.677751 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d20ab-d23e-41e3-a933-d8eac975fdb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:10.678033 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d20ab-d23e-41e3-a933-d8eac975fdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:10.678308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d20ab-d23e-41e3-a933-d8eac975fdb1 admin admin] PATH_INFO: `/v3/projects/4f0d1a97ee4348878a4e7adf6337e04e/users/ef5da9f384a54e8e8e2f785847bcf29d/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:10.695379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d20ab-d23e-41e3-a933-d8eac975fdb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f0d1a97ee4348878a4e7adf6337e04e, user_id=ef5da9f384a54e8e8e2f785847bcf29d, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:10.698424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d20ab-d23e-41e3-a933-d8eac975fdb1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:10.797174 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1193/2356] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:22:10 2022] PUT /identity/v3/projects/4f0d1a97ee4348878a4e7adf6337e04e/users/ef5da9f384a54e8e8e2f785847bcf29d/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:22:10.808752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddb8c8-099a-4c06-9066-ae29b2bdb624 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:10.808978 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddb8c8-099a-4c06-9066-ae29b2bdb624 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:10.809248 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddb8c8-099a-4c06-9066-ae29b2bdb624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:10.840278 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-b7ddb8c8-099a-4c06-9066-ae29b2bdb624 None None] MFA Rules not processed for user `ef5da9f384a54e8e8e2f785847bcf29d`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:10.841503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b7ddb8c8-099a-4c06-9066-ae29b2bdb624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:10.909216 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1164/2357] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:22:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:22:10.921378 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d20ab-d23e-41e3-a933-d8eac975fdb1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:10.923670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-669d20ab-d23e-41e3-a933-d8eac975fdb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:10.979871 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d20ab-d23e-41e3-a933-d8eac975fdb1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:10.984057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2827b-2314-4458-88a5-de732b71157c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:10.984708 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2827b-2314-4458-88a5-de732b71157c service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:10.984985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2827b-2314-4458-88a5-de732b71157c service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:10.985253 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2827b-2314-4458-88a5-de732b71157c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:10.993343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d2827b-2314-4458-88a5-de732b71157c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:10.994759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d2827b-2314-4458-88a5-de732b71157c service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:11.004091 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1194/2358] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:22:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:22:14.415892 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1b9a4-c6a7-41ee-9f57-82829a32450f None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:14.416363 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1b9a4-c6a7-41ee-9f57-82829a32450f None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:14.416468 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1b9a4-c6a7-41ee-9f57-82829a32450f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:14.435694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-96d1b9a4-c6a7-41ee-9f57-82829a32450f None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:14.437411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-96d1b9a4-c6a7-41ee-9f57-82829a32450f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:14.505929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1165/2359] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Oct 6 08:22:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:22:14.651232 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2827b-2314-4458-88a5-de732b71157c service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:14.652747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d3d2827b-2314-4458-88a5-de732b71157c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:14.686459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:22:14.720507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2827b-2314-4458-88a5-de732b71157c service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:14.726219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd37e1-804f-4020-9d31-06074c445aa5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:14.726823 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd37e1-804f-4020-9d31-06074c445aa5 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:14.727155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd37e1-804f-4020-9d31-06074c445aa5 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:14.727428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd37e1-804f-4020-9d31-06074c445aa5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:14.736186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd37e1-804f-4020-9d31-06074c445aa5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:14.737922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd37e1-804f-4020-9d31-06074c445aa5 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:14.746166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1195/2360] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:22:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:22:24.498793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:22:24.779710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=18 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:22:26.481454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1b9a4-c6a7-41ee-9f57-82829a32450f None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:26.484054 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-96d1b9a4-c6a7-41ee-9f57-82829a32450f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:26.609976 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1b9a4-c6a7-41ee-9f57-82829a32450f None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:26.617237 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caff56a-7e53-4145-b359-48ded373ff8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6074872303d3468fb566378fa34e8bea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:26.617868 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caff56a-7e53-4145-b359-48ded373ff8e service placement] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:26.618153 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caff56a-7e53-4145-b359-48ded373ff8e service placement] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:26.618429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caff56a-7e53-4145-b359-48ded373ff8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:26.620313 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-8caff56a-7e53-4145-b359-48ded373ff8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:26.635217 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caff56a-7e53-4145-b359-48ded373ff8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:26.641602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caff56a-7e53-4145-b359-48ded373ff8e service placement] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:26.650386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1166/2361] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:22:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:22:32.495388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e3064-0ccc-4c3f-ba53-2e939ea51019 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:32.499609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e3064-0ccc-4c3f-ba53-2e939ea51019 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:32.499609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e3064-0ccc-4c3f-ba53-2e939ea51019 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:32.499609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1196/2362] 10.222.0.30 () {54 vars in 921 bytes} [Thu Oct 6 08:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:22:32.505993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b3aec-1707-4a34-b126-62cc9edb5c79 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:32.506478 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b3aec-1707-4a34-b126-62cc9edb5c79 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:32.506574 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b3aec-1707-4a34-b126-62cc9edb5c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:32.547395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-2d2b3aec-1707-4a34-b126-62cc9edb5c79 None None] MFA Rules not processed for user `209795285384486183cda455fe316bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:32.551440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-2d2b3aec-1707-4a34-b126-62cc9edb5c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:32.624234 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1167/2363] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Oct 6 08:22:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:22:32.636504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e3064-0ccc-4c3f-ba53-2e939ea51019 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:32.638686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2d4e3064-0ccc-4c3f-ba53-2e939ea51019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:32.698230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e3064-0ccc-4c3f-ba53-2e939ea51019 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:32.701808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c6a2a-8bf4-411b-9177-a80cd0d77772 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30909d152a5c4a1fac6c1a2765868177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:32.702419 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c6a2a-8bf4-411b-9177-a80cd0d77772 service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:32.702797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c6a2a-8bf4-411b-9177-a80cd0d77772 service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:32.703045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c6a2a-8bf4-411b-9177-a80cd0d77772 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:32.714214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c6a2a-8bf4-411b-9177-a80cd0d77772 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:32.715656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c6a2a-8bf4-411b-9177-a80cd0d77772 service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:32.727421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1197/2364] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:22:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:22:32.934972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56de1d6-0192-4041-8cc3-37c9dc7a732d None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:32.935265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56de1d6-0192-4041-8cc3-37c9dc7a732d None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:32.935584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56de1d6-0192-4041-8cc3-37c9dc7a732d None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:32.936936 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1168/2365] 10.222.0.30 () {54 vars in 921 bytes} [Thu Oct 6 08:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:22:32.942490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd904fe1-adb0-4cfd-b47a-ef6f13865972 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:32.943568 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd904fe1-adb0-4cfd-b47a-ef6f13865972 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:32.943851 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd904fe1-adb0-4cfd-b47a-ef6f13865972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:32.966614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bd904fe1-adb0-4cfd-b47a-ef6f13865972 None None] MFA Rules not processed for user `bf6798f89acf44e897c481924edd9047`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:32.967835 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bd904fe1-adb0-4cfd-b47a-ef6f13865972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:33.021963 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1198/2366] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Oct 6 08:22:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:22:33.034271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56de1d6-0192-4041-8cc3-37c9dc7a732d None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:33.039595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f56de1d6-0192-4041-8cc3-37c9dc7a732d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:33.049698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56de1d6-0192-4041-8cc3-37c9dc7a732d None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:33.053498 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf55a4-7f89-449d-9e61-3d0eb7f56c6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:33.054035 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf55a4-7f89-449d-9e61-3d0eb7f56c6a service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:33.054308 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf55a4-7f89-449d-9e61-3d0eb7f56c6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:33.054576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf55a4-7f89-449d-9e61-3d0eb7f56c6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:33.061830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf55a4-7f89-449d-9e61-3d0eb7f56c6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:33.064407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf55a4-7f89-449d-9e61-3d0eb7f56c6a service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:33.074174 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a228ec9-3a65-4ca9-af60-1c3224b96b43 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:33.074456 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a228ec9-3a65-4ca9-af60-1c3224b96b43 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:33.074753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a228ec9-3a65-4ca9-af60-1c3224b96b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:33.077116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1169/2367] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:22:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:22:33.094252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9a228ec9-3a65-4ca9-af60-1c3224b96b43 None None] MFA Rules not processed for user `9932c7fb57394d709a27023f3ae2677e`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:33.095527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9a228ec9-3a65-4ca9-af60-1c3224b96b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:33.154475 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1199/2368] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:22:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:22:33.165708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbc73f-986b-4819-b0a0-ffedd5d72a10 None None] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:33.165708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbc73f-986b-4819-b0a0-ffedd5d72a10 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:33.166128 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbc73f-986b-4819-b0a0-ffedd5d72a10 None None] PATH_INFO: `/` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:33.167543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1170/2369] 10.222.0.30 () {54 vars in 921 bytes} [Thu Oct 6 08:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 06 08:22:33.173400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4945f4f-f8b1-4f6f-a1a8-64317ed26bc3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:33.173797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4945f4f-f8b1-4f6f-a1a8-64317ed26bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:33.174064 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4945f4f-f8b1-4f6f-a1a8-64317ed26bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:33.194735 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d4945f4f-f8b1-4f6f-a1a8-64317ed26bc3 None None] MFA Rules not processed for user `209795285384486183cda455fe316bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:33.195952 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d4945f4f-f8b1-4f6f-a1a8-64317ed26bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:33.206449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1200/2370] 10.222.0.30 () {58 vars in 1018 bytes} [Thu Oct 6 08:22:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:22:33.217842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbc73f-986b-4819-b0a0-ffedd5d72a10 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:33.219702 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9dfbc73f-986b-4819-b0a0-ffedd5d72a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:33.233116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbc73f-986b-4819-b0a0-ffedd5d72a10 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:33.237539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f809b-2197-4c06-bdf6-b0a1689a62af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:33.238206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f809b-2197-4c06-bdf6-b0a1689a62af service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:33.238566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f809b-2197-4c06-bdf6-b0a1689a62af service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:33.238918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f809b-2197-4c06-bdf6-b0a1689a62af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:33.249825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f809b-2197-4c06-bdf6-b0a1689a62af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:33.253260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f809b-2197-4c06-bdf6-b0a1689a62af service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:33.263537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1171/2371] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:22:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:22:33.478114 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4945f4f-f8b1-4f6f-a1a8-64317ed26bc3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:33.488307 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4945f4f-f8b1-4f6f-a1a8-64317ed26bc3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:33.492093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f81b76-de8f-422b-8667-d481d5502508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:33.492691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f81b76-de8f-422b-8667-d481d5502508 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:33.492965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f81b76-de8f-422b-8667-d481d5502508 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:33.493230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f81b76-de8f-422b-8667-d481d5502508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:33.495040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-92f81b76-de8f-422b-8667-d481d5502508 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:33.505026 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f81b76-de8f-422b-8667-d481d5502508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:33.506438 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f81b76-de8f-422b-8667-d481d5502508 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:33.514582 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1201/2372] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:22:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:22:38.279341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d79d76-1c3f-4c8b-9fd1-f1e2b8025fab None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:38.279809 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d79d76-1c3f-4c8b-9fd1-f1e2b8025fab None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:38.280016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d79d76-1c3f-4c8b-9fd1-f1e2b8025fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:38.306827 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-a6d79d76-1c3f-4c8b-9fd1-f1e2b8025fab None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:38.308399 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a6d79d76-1c3f-4c8b-9fd1-f1e2b8025fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:38.318467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1172/2373] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Oct 6 08:22:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:22:38.334202 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f81b76-de8f-422b-8667-d481d5502508 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:38.349389 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f81b76-de8f-422b-8667-d481d5502508 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:38.356598 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476c6e0-3135-4f5b-aa01-b100059d8394 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:38.357353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476c6e0-3135-4f5b-aa01-b100059d8394 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:38.357690 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476c6e0-3135-4f5b-aa01-b100059d8394 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:38.358018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476c6e0-3135-4f5b-aa01-b100059d8394 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:38.368421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476c6e0-3135-4f5b-aa01-b100059d8394 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:38.370142 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476c6e0-3135-4f5b-aa01-b100059d8394 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:38.380292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1202/2374] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:22:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:22:42.256720 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d79d76-1c3f-4c8b-9fd1-f1e2b8025fab None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:42.268511 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:22:42.329848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d79d76-1c3f-4c8b-9fd1-f1e2b8025fab None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:42.335592 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e2684-a7df-4e91-b162-a64db2afcdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:42.336239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e2684-a7df-4e91-b162-a64db2afcdd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:42.336571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e2684-a7df-4e91-b162-a64db2afcdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:42.336791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e2684-a7df-4e91-b162-a64db2afcdd9 admin admin] PATH_INFO: `/v3/users/ef5da9f384a54e8e8e2f785847bcf29d` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:42.337940 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e2684-a7df-4e91-b162-a64db2afcdd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef5da9f384a54e8e8e2f785847bcf29d)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:42.340017 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e2684-a7df-4e91-b162-a64db2afcdd9 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:42.561210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3e3e2684-a7df-4e91-b162-a64db2afcdd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef5da9f384a54e8e8e2f785847bcf29d'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:42.561874 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3e3e2684-a7df-4e91-b162-a64db2afcdd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef5da9f384a54e8e8e2f785847bcf29d'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:42.575666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3e3e2684-a7df-4e91-b162-a64db2afcdd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef5da9f384a54e8e8e2f785847bcf29d'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:42.652639 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3e3e2684-a7df-4e91-b162-a64db2afcdd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef5da9f384a54e8e8e2f785847bcf29d'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:42.653338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3e3e2684-a7df-4e91-b162-a64db2afcdd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef5da9f384a54e8e8e2f785847bcf29d'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:42.668432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1173/2375] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:22:42 2022] DELETE /identity/v3/users/ef5da9f384a54e8e8e2f785847bcf29d => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:22:42.673268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd839d56-13f3-467a-9ab2-7e7b20e2cead None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:42.680682 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd839d56-13f3-467a-9ab2-7e7b20e2cead None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:42.680682 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd839d56-13f3-467a-9ab2-7e7b20e2cead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:42.695264 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fd839d56-13f3-467a-9ab2-7e7b20e2cead None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:42.696544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fd839d56-13f3-467a-9ab2-7e7b20e2cead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:42.720918 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:22:42.769544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1203/2376] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:22:42.779746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e2684-a7df-4e91-b162-a64db2afcdd9 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:42.781161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3e3e2684-a7df-4e91-b162-a64db2afcdd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:42.841125 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e2684-a7df-4e91-b162-a64db2afcdd9 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:42.845529 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae921e6-dcd1-4880-a9ad-b9149e25e73e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:42.846205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae921e6-dcd1-4880-a9ad-b9149e25e73e service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:42.846554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae921e6-dcd1-4880-a9ad-b9149e25e73e service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:42.846905 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae921e6-dcd1-4880-a9ad-b9149e25e73e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:42.857320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae921e6-dcd1-4880-a9ad-b9149e25e73e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:42.858951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae921e6-dcd1-4880-a9ad-b9149e25e73e service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:42.869757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1174/2377] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:22:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:22:43.414006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd839d56-13f3-467a-9ab2-7e7b20e2cead None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:43.427117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fd839d56-13f3-467a-9ab2-7e7b20e2cead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:43.453110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd839d56-13f3-467a-9ab2-7e7b20e2cead None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:43.462982 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbcb6e-ed69-44cb-9e97-0a60bd8a1113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:43.475825 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbcb6e-ed69-44cb-9e97-0a60bd8a1113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:43.476971 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbcb6e-ed69-44cb-9e97-0a60bd8a1113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:43.477333 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbcb6e-ed69-44cb-9e97-0a60bd8a1113 admin admin] PATH_INFO: `/v3/projects/4f0d1a97ee4348878a4e7adf6337e04e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:43.478773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbcb6e-ed69-44cb-9e97-0a60bd8a1113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f0d1a97ee4348878a4e7adf6337e04e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:43.481411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbcb6e-ed69-44cb-9e97-0a60bd8a1113 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:43.659952 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-eebbcb6e-ed69-44cb-9e97-0a60bd8a1113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f0d1a97ee4348878a4e7adf6337e04e', 'request_id': 'req-eebbcb6e-ed69-44cb-9e97-0a60bd8a1113'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:43.666066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-eebbcb6e-ed69-44cb-9e97-0a60bd8a1113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f0d1a97ee4348878a4e7adf6337e04e', 'request_id': 'req-eebbcb6e-ed69-44cb-9e97-0a60bd8a1113'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:43.726878 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-eebbcb6e-ed69-44cb-9e97-0a60bd8a1113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f0d1a97ee4348878a4e7adf6337e04e', 'request_id': 'req-eebbcb6e-ed69-44cb-9e97-0a60bd8a1113'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:43.743316 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-eebbcb6e-ed69-44cb-9e97-0a60bd8a1113 admin admin] The token cache is being invalidate because project 4f0d1a97ee4348878a4e7adf6337e04e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:22:43.743882 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-eebbcb6e-ed69-44cb-9e97-0a60bd8a1113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:43.744694 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-eebbcb6e-ed69-44cb-9e97-0a60bd8a1113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:22:43.746697 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1204/2378] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:22:43 2022] DELETE /identity/v3/projects/4f0d1a97ee4348878a4e7adf6337e04e => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:22:43.755973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6194ebb-dbdb-4256-a70f-30f064b87a4d None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:43.756588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6194ebb-dbdb-4256-a70f-30f064b87a4d None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:43.756702 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6194ebb-dbdb-4256-a70f-30f064b87a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:43.788775 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-c6194ebb-dbdb-4256-a70f-30f064b87a4d None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:43.790029 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-c6194ebb-dbdb-4256-a70f-30f064b87a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:43.826089 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:22:43.863435 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1175/2379] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:22:43.871212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b49bde-1cca-42cb-a4c3-3e82e4502955 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:43.871494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b49bde-1cca-42cb-a4c3-3e82e4502955 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:43.871770 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b49bde-1cca-42cb-a4c3-3e82e4502955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:43.895112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-29b49bde-1cca-42cb-a4c3-3e82e4502955 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:43.900309 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-29b49bde-1cca-42cb-a4c3-3e82e4502955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:43.904561 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1205/2380] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:22:43.909548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6194ebb-dbdb-4256-a70f-30f064b87a4d None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:43.919768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6194ebb-dbdb-4256-a70f-30f064b87a4d None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:43.925182 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a722c8-300a-4a4c-bb16-ac91d30d9bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:43.929072 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a722c8-300a-4a4c-bb16-ac91d30d9bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:43.929072 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a722c8-300a-4a4c-bb16-ac91d30d9bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:43.929072 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a722c8-300a-4a4c-bb16-ac91d30d9bc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:43.929072 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a722c8-300a-4a4c-bb16-ac91d30d9bc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:22:43.929072 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a722c8-300a-4a4c-bb16-ac91d30d9bc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:43.929072 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a722c8-300a-4a4c-bb16-ac91d30d9bc8 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:43.940867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1176/2381] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:22:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:22:43.945174 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b49bde-1cca-42cb-a4c3-3e82e4502955 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:43.957624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b49bde-1cca-42cb-a4c3-3e82e4502955 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:43.965113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894599f-dfc3-4a28-8e6d-430005f739fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:43.965729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894599f-dfc3-4a28-8e6d-430005f739fb admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:43.965995 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894599f-dfc3-4a28-8e6d-430005f739fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:43.966267 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894599f-dfc3-4a28-8e6d-430005f739fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:43.966905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894599f-dfc3-4a28-8e6d-430005f739fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:43.968953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894599f-dfc3-4a28-8e6d-430005f739fb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:44.055571 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1206/2382] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:22:43 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:22:44.059983 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a722c8-300a-4a4c-bb16-ac91d30d9bc8 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:44.072050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a722c8-300a-4a4c-bb16-ac91d30d9bc8 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:44.147633 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4593ce-2345-42e2-8c96-4b694efa182b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:44.153000 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4593ce-2345-42e2-8c96-4b694efa182b admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:44.153000 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4593ce-2345-42e2-8c96-4b694efa182b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:44.153000 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4593ce-2345-42e2-8c96-4b694efa182b admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:44.153000 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4593ce-2345-42e2-8c96-4b694efa182b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:44.153000 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4593ce-2345-42e2-8c96-4b694efa182b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:44.303829 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1177/2383] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:22:44 2022] POST /identity/v3/users => generated 287 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:22:44.307994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894599f-dfc3-4a28-8e6d-430005f739fb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:44.319320 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894599f-dfc3-4a28-8e6d-430005f739fb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:44.324323 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95863aa0-a849-4b70-b1c9-e02ffd805e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:44.324930 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95863aa0-a849-4b70-b1c9-e02ffd805e86 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:44.325228 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95863aa0-a849-4b70-b1c9-e02ffd805e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:44.325524 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95863aa0-a849-4b70-b1c9-e02ffd805e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:44.326181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95863aa0-a849-4b70-b1c9-e02ffd805e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:44.327828 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95863aa0-a849-4b70-b1c9-e02ffd805e86 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:44.342476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1207/2384] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:22:44.346827 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4593ce-2345-42e2-8c96-4b694efa182b admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:44.357546 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4593ce-2345-42e2-8c96-4b694efa182b admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:44.362750 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4926d7-a80a-4f0f-974e-006ecc62ce0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:44.363380 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4926d7-a80a-4f0f-974e-006ecc62ce0c admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:44.363654 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4926d7-a80a-4f0f-974e-006ecc62ce0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:44.363921 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4926d7-a80a-4f0f-974e-006ecc62ce0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:44.364578 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4926d7-a80a-4f0f-974e-006ecc62ce0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:44.366169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4926d7-a80a-4f0f-974e-006ecc62ce0c admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:44.377312 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1178/2385] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:22:44.381684 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95863aa0-a849-4b70-b1c9-e02ffd805e86 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:44.392416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95863aa0-a849-4b70-b1c9-e02ffd805e86 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:44.397043 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973d6b2-8921-4556-b3ba-8b17a54ee6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:44.397758 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973d6b2-8921-4556-b3ba-8b17a54ee6f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:44.398078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973d6b2-8921-4556-b3ba-8b17a54ee6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:44.398353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973d6b2-8921-4556-b3ba-8b17a54ee6f6 admin admin] PATH_INFO: `/v3/projects/f0458e73970e41b3ac7948f8eb8f75e9/users/b6fd97062022478e9d34c318b309fd90/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:44.413413 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973d6b2-8921-4556-b3ba-8b17a54ee6f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0458e73970e41b3ac7948f8eb8f75e9, user_id=b6fd97062022478e9d34c318b309fd90, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:44.417981 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973d6b2-8921-4556-b3ba-8b17a54ee6f6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:44.540205 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1208/2386] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:22:44 2022] PUT /identity/v3/projects/f0458e73970e41b3ac7948f8eb8f75e9/users/b6fd97062022478e9d34c318b309fd90/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:22:44.545450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6564b03-b388-40fc-baa9-86d0f0e4411b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:44.545739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6564b03-b388-40fc-baa9-86d0f0e4411b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:44.546020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6564b03-b388-40fc-baa9-86d0f0e4411b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:44.568249 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-c6564b03-b388-40fc-baa9-86d0f0e4411b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:44.569479 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-c6564b03-b388-40fc-baa9-86d0f0e4411b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:44.647454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1179/2387] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:22:44.683624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973d6b2-8921-4556-b3ba-8b17a54ee6f6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:44.685819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b973d6b2-8921-4556-b3ba-8b17a54ee6f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:44.801266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:22:44.807355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973d6b2-8921-4556-b3ba-8b17a54ee6f6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:44.816272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616581c3-2f42-4f53-a5a2-4d29409482a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:44.816272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616581c3-2f42-4f53-a5a2-4d29409482a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:44.816272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616581c3-2f42-4f53-a5a2-4d29409482a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:44.816272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616581c3-2f42-4f53-a5a2-4d29409482a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:44.823091 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616581c3-2f42-4f53-a5a2-4d29409482a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:44.824567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616581c3-2f42-4f53-a5a2-4d29409482a9 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:44.837489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1209/2388] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:22:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:22:46.261885 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50091f0b-ec56-474e-850f-7c0fc5f78ae5 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:46.262435 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50091f0b-ec56-474e-850f-7c0fc5f78ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:46.262519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50091f0b-ec56-474e-850f-7c0fc5f78ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:46.300244 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-50091f0b-ec56-474e-850f-7c0fc5f78ae5 None None] MFA Rules not processed for user `b6fd97062022478e9d34c318b309fd90`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:46.301536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-50091f0b-ec56-474e-850f-7c0fc5f78ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:46.346262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:22:46.396900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1180/2389] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:22:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:22:46.408733 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616581c3-2f42-4f53-a5a2-4d29409482a9 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:46.411665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-616581c3-2f42-4f53-a5a2-4d29409482a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:46.510816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616581c3-2f42-4f53-a5a2-4d29409482a9 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:46.515217 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09301f-55cd-463f-b204-eb274255ebbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:46.515899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09301f-55cd-463f-b204-eb274255ebbe service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:46.516197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09301f-55cd-463f-b204-eb274255ebbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:46.516596 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09301f-55cd-463f-b204-eb274255ebbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:46.527331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09301f-55cd-463f-b204-eb274255ebbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:46.528868 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09301f-55cd-463f-b204-eb274255ebbe service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:46.541871 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1210/2390] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:22:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:22:49.732791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cceea-9916-48be-8276-1c7746e333c9 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:49.733335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cceea-9916-48be-8276-1c7746e333c9 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:49.733335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cceea-9916-48be-8276-1c7746e333c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:49.756053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-042cceea-9916-48be-8276-1c7746e333c9 None None] MFA Rules not processed for user `b6fd97062022478e9d34c318b309fd90`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:49.757268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-042cceea-9916-48be-8276-1c7746e333c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:49.764460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1181/2391] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:22:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:22:56.534615 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=32 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:22:58.717027 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7ba14-f9d8-43f8-a612-70118fdeaec4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:58.718054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7ba14-f9d8-43f8-a612-70118fdeaec4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:58.718613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7ba14-f9d8-43f8-a612-70118fdeaec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:58.742514 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ffa7ba14-f9d8-43f8-a612-70118fdeaec4 None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:22:58.744895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ffa7ba14-f9d8-43f8-a612-70118fdeaec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:58.805630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1211/2392] 10.222.0.30 () {58 vars in 967 bytes} [Thu Oct 6 08:22:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:22:58.822885 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cceea-9916-48be-8276-1c7746e333c9 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:22:58.824465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-042cceea-9916-48be-8276-1c7746e333c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:22:58.904910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cceea-9916-48be-8276-1c7746e333c9 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:22:58.911317 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bf072-b6c1-46a2-aacd-d178d74e75c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:22:58.911934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bf072-b6c1-46a2-aacd-d178d74e75c3 service swift] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:22:58.912213 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bf072-b6c1-46a2-aacd-d178d74e75c3 service swift] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:22:58.912484 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bf072-b6c1-46a2-aacd-d178d74e75c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:22:58.921284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9bf072-b6c1-46a2-aacd-d178d74e75c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:22:58.922880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9bf072-b6c1-46a2-aacd-d178d74e75c3 service swift] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:22:58.932032 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1182/2393] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:22:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:23:08.797603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=25 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:23:08.928653 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=50 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:23:13.632076 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e4590-3e74-4eb1-887e-881f5a24fda5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:23:13.632535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e4590-3e74-4eb1-887e-881f5a24fda5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:23:13.632628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e4590-3e74-4eb1-887e-881f5a24fda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:23:13.651609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2f4e4590-3e74-4eb1-887e-881f5a24fda5 None None] MFA Rules not processed for user `b6fd97062022478e9d34c318b309fd90`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:23:13.652793 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2f4e4590-3e74-4eb1-887e-881f5a24fda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:23:13.660076 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1212/2394] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:23:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:23:23.646814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:23:25.186051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bf072-b6c1-46a2-aacd-d178d74e75c3 service swift] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:23:25.187490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-fe9bf072-b6c1-46a2-aacd-d178d74e75c3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:23:25.200020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bf072-b6c1-46a2-aacd-d178d74e75c3 service swift] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:23:25.203788 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4dcd2-d757-4c00-87a5-c199f47a60d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:23:25.204407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4dcd2-d757-4c00-87a5-c199f47a60d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:23:25.204770 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4dcd2-d757-4c00-87a5-c199f47a60d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:23:25.205041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4dcd2-d757-4c00-87a5-c199f47a60d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:23:25.206806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f8f4dcd2-d757-4c00-87a5-c199f47a60d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:23:25.280613 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4dcd2-d757-4c00-87a5-c199f47a60d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:23:25.282062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4dcd2-d757-4c00-87a5-c199f47a60d7 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:23:25.293427 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1183/2395] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:23:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:23:26.136313 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e4590-3e74-4eb1-887e-881f5a24fda5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:23:26.152419 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e4590-3e74-4eb1-887e-881f5a24fda5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:23:26.156111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263191bb-5baa-4b64-b165-219b9bbaba7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:23:26.156717 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263191bb-5baa-4b64-b165-219b9bbaba7f service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:23:26.156991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263191bb-5baa-4b64-b165-219b9bbaba7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:23:26.157258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263191bb-5baa-4b64-b165-219b9bbaba7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:23:26.159018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-263191bb-5baa-4b64-b165-219b9bbaba7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:23:26.173201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263191bb-5baa-4b64-b165-219b9bbaba7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:23:26.174683 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263191bb-5baa-4b64-b165-219b9bbaba7f service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:23:26.186818 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1213/2396] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:23:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:23:35.288389 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=34 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:23:36.182531 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:23:39.480633 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810684d-e6ed-48bb-8167-7116655c946b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:23:39.481083 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810684d-e6ed-48bb-8167-7116655c946b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:23:39.481259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810684d-e6ed-48bb-8167-7116655c946b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:23:39.502790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-f810684d-e6ed-48bb-8167-7116655c946b None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:23:39.502790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f810684d-e6ed-48bb-8167-7116655c946b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:23:39.510925 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1184/2397] 10.222.0.30 () {58 vars in 1009 bytes} [Thu Oct 6 08:23:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:23:39.533764 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263191bb-5baa-4b64-b165-219b9bbaba7f service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:23:39.545672 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263191bb-5baa-4b64-b165-219b9bbaba7f service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:23:39.551270 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7440ee-0bc0-4de0-8bfd-0eeb8d3798c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:23:39.551841 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7440ee-0bc0-4de0-8bfd-0eeb8d3798c3 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:23:39.552115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7440ee-0bc0-4de0-8bfd-0eeb8d3798c3 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:23:39.552386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7440ee-0bc0-4de0-8bfd-0eeb8d3798c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:23:39.560752 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7440ee-0bc0-4de0-8bfd-0eeb8d3798c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:23:39.562329 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7440ee-0bc0-4de0-8bfd-0eeb8d3798c3 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:23:39.571130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1214/2398] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:23:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:23:49.495525 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:23:49.567585 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:23:56.497043 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810684d-e6ed-48bb-8167-7116655c946b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:23:56.507805 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810684d-e6ed-48bb-8167-7116655c946b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:23:56.511587 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444551a-8c43-49d1-9428-dc25103f7da9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:23:56.512187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444551a-8c43-49d1-9428-dc25103f7da9 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:23:56.512485 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444551a-8c43-49d1-9428-dc25103f7da9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:23:56.512753 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444551a-8c43-49d1-9428-dc25103f7da9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:23:56.514709 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6444551a-8c43-49d1-9428-dc25103f7da9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:23:56.526545 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444551a-8c43-49d1-9428-dc25103f7da9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:23:56.527987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444551a-8c43-49d1-9428-dc25103f7da9 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:23:56.537410 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1185/2399] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:23:57.605317 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736f7b5-6b57-4c43-a319-7a36f1d56bba None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:23:57.605803 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736f7b5-6b57-4c43-a319-7a36f1d56bba None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:23:57.605901 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736f7b5-6b57-4c43-a319-7a36f1d56bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:23:57.657230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d736f7b5-6b57-4c43-a319-7a36f1d56bba None None] MFA Rules not processed for user `0bd9c73ff2d94089a34de12c1155e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:23:57.658387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d736f7b5-6b57-4c43-a319-7a36f1d56bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:23:57.676164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1215/2400] 10.222.0.30 () {58 vars in 967 bytes} [Thu Oct 6 08:23:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:23:57.688712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444551a-8c43-49d1-9428-dc25103f7da9 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:23:57.711289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444551a-8c43-49d1-9428-dc25103f7da9 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:23:57.716296 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b9d79-b804-4f3c-ba60-8e2fa5c73279 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fa1c430c9f7416eaf1d17300b827e05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:23:57.716965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b9d79-b804-4f3c-ba60-8e2fa5c73279 service swift] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:23:57.717311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b9d79-b804-4f3c-ba60-8e2fa5c73279 service swift] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:23:57.717584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b9d79-b804-4f3c-ba60-8e2fa5c73279 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:23:57.726233 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b9d79-b804-4f3c-ba60-8e2fa5c73279 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:23:57.727841 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b9d79-b804-4f3c-ba60-8e2fa5c73279 service swift] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:23:57.735861 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1186/2401] 10.222.0.30 () {60 vars in 1384 bytes} [Thu Oct 6 08:23:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 06 08:23:59.830036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736f7b5-6b57-4c43-a319-7a36f1d56bba None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:23:59.832211 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d736f7b5-6b57-4c43-a319-7a36f1d56bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:23:59.905656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736f7b5-6b57-4c43-a319-7a36f1d56bba None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:23:59.910838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b0f14-e052-411c-b1d9-a9007a94411a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6074872303d3468fb566378fa34e8bea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:23:59.911474 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b0f14-e052-411c-b1d9-a9007a94411a service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:23:59.911751 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b0f14-e052-411c-b1d9-a9007a94411a service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:23:59.912016 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b0f14-e052-411c-b1d9-a9007a94411a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:23:59.913745 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-541b0f14-e052-411c-b1d9-a9007a94411a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:23:59.924979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b0f14-e052-411c-b1d9-a9007a94411a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:23:59.926587 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b0f14-e052-411c-b1d9-a9007a94411a service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:23:59.936275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1216/2402] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:23:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:23:59.969950 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b9d79-b804-4f3c-ba60-8e2fa5c73279 service swift] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:23:59.970857 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4e5b9d79-b804-4f3c-ba60-8e2fa5c73279 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:23:59.984872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b9d79-b804-4f3c-ba60-8e2fa5c73279 service swift] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:23:59.989988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e87ac-3d8c-4006-af03-4aff970647a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:23:59.990618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e87ac-3d8c-4006-af03-4aff970647a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:23:59.990895 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e87ac-3d8c-4006-af03-4aff970647a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:23:59.991205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e87ac-3d8c-4006-af03-4aff970647a6 admin admin] PATH_INFO: `/v3/users/cb73bbd331754280a2a0cd64290ec3c8` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:00.005727 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e87ac-3d8c-4006-af03-4aff970647a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb73bbd331754280a2a0cd64290ec3c8)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:00.007956 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e87ac-3d8c-4006-af03-4aff970647a6 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:00.201366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fd2e87ac-3d8c-4006-af03-4aff970647a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb73bbd331754280a2a0cd64290ec3c8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:00.202055 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fd2e87ac-3d8c-4006-af03-4aff970647a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb73bbd331754280a2a0cd64290ec3c8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:00.219675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fd2e87ac-3d8c-4006-af03-4aff970647a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb73bbd331754280a2a0cd64290ec3c8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:00.381847 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fd2e87ac-3d8c-4006-af03-4aff970647a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb73bbd331754280a2a0cd64290ec3c8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:00.382442 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fd2e87ac-3d8c-4006-af03-4aff970647a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb73bbd331754280a2a0cd64290ec3c8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:00.397414 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1187/2403] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:23:59 2022] DELETE /identity/v3/users/cb73bbd331754280a2a0cd64290ec3c8 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:00.687512 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b0f14-e052-411c-b1d9-a9007a94411a service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:00.706457 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b0f14-e052-411c-b1d9-a9007a94411a service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:00.764068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df29cc-3518-4101-ab62-9a7afd7dedf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:00.764712 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df29cc-3518-4101-ab62-9a7afd7dedf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:00.764981 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df29cc-3518-4101-ab62-9a7afd7dedf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:00.765244 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df29cc-3518-4101-ab62-9a7afd7dedf7 admin admin] PATH_INFO: `/v3/projects/73f960dc36054e9ba7003628a82b7039` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:00.775514 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df29cc-3518-4101-ab62-9a7afd7dedf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73f960dc36054e9ba7003628a82b7039)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:00.777559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df29cc-3518-4101-ab62-9a7afd7dedf7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:00.793231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e87ac-3d8c-4006-af03-4aff970647a6 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:00.793231 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-fd2e87ac-3d8c-4006-af03-4aff970647a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:00.807703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e87ac-3d8c-4006-af03-4aff970647a6 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:00.812820 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba94d43-2d70-4b26-805b-b86f3e909b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:00.813535 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba94d43-2d70-4b26-805b-b86f3e909b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:00.813821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba94d43-2d70-4b26-805b-b86f3e909b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:00.814097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba94d43-2d70-4b26-805b-b86f3e909b17 admin admin] PATH_INFO: `/v3/users/b6fd97062022478e9d34c318b309fd90` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:00.815260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba94d43-2d70-4b26-805b-b86f3e909b17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6fd97062022478e9d34c318b309fd90)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:00.817202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba94d43-2d70-4b26-805b-b86f3e909b17 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:00.883979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:00.887628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-91df29cc-3518-4101-ab62-9a7afd7dedf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73f960dc36054e9ba7003628a82b7039', 'request_id': 'req-91df29cc-3518-4101-ab62-9a7afd7dedf7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:00.894979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-91df29cc-3518-4101-ab62-9a7afd7dedf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73f960dc36054e9ba7003628a82b7039', 'request_id': 'req-91df29cc-3518-4101-ab62-9a7afd7dedf7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:00.965625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-91df29cc-3518-4101-ab62-9a7afd7dedf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73f960dc36054e9ba7003628a82b7039', 'request_id': 'req-91df29cc-3518-4101-ab62-9a7afd7dedf7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:00.976525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-91df29cc-3518-4101-ab62-9a7afd7dedf7 admin admin] The token cache is being invalidate because project 73f960dc36054e9ba7003628a82b7039 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:00.976809 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-91df29cc-3518-4101-ab62-9a7afd7dedf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:00.977435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-91df29cc-3518-4101-ab62-9a7afd7dedf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:00.978919 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1217/2404] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:00 2022] DELETE /identity/v3/projects/73f960dc36054e9ba7003628a82b7039 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:00.983553 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca783b-2bc1-4735-abbc-e8ba5dc8ce06 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:00.983836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca783b-2bc1-4735-abbc-e8ba5dc8ce06 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:00.984103 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca783b-2bc1-4735-abbc-e8ba5dc8ce06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:01.004324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a5ca783b-2bc1-4735-abbc-e8ba5dc8ce06 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:01.005518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a5ca783b-2bc1-4735-abbc-e8ba5dc8ce06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:01.048516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fba94d43-2d70-4b26-805b-b86f3e909b17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6fd97062022478e9d34c318b309fd90'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:01.049188 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fba94d43-2d70-4b26-805b-b86f3e909b17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6fd97062022478e9d34c318b309fd90'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:01.062478 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fba94d43-2d70-4b26-805b-b86f3e909b17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6fd97062022478e9d34c318b309fd90'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:01.064707 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1218/2405] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:24:01.073919 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d58417-209d-4784-8720-dbd9fdcbf60c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:01.074201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d58417-209d-4784-8720-dbd9fdcbf60c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:01.074475 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d58417-209d-4784-8720-dbd9fdcbf60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:01.093516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a0d58417-209d-4784-8720-dbd9fdcbf60c None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:01.094701 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a0d58417-209d-4784-8720-dbd9fdcbf60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:01.102613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1219/2406] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:24:01.106486 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d58417-209d-4784-8720-dbd9fdcbf60c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:01.115887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d58417-209d-4784-8720-dbd9fdcbf60c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:01.120421 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c877d6-e026-431a-8857-95837c6048d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:01.120973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c877d6-e026-431a-8857-95837c6048d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:01.121246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c877d6-e026-431a-8857-95837c6048d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:01.121513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c877d6-e026-431a-8857-95837c6048d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:01.122134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c877d6-e026-431a-8857-95837c6048d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:24:01.122429 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c877d6-e026-431a-8857-95837c6048d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:01.124035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c877d6-e026-431a-8857-95837c6048d7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:01.135835 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1220/2407] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 08:24:01.140379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c877d6-e026-431a-8857-95837c6048d7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:01.150143 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c877d6-e026-431a-8857-95837c6048d7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:01.154865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d10eec-645b-4505-b69a-4ac7d94de5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:01.155478 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d10eec-645b-4505-b69a-4ac7d94de5e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:01.155748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d10eec-645b-4505-b69a-4ac7d94de5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:01.156012 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d10eec-645b-4505-b69a-4ac7d94de5e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:01.156631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d10eec-645b-4505-b69a-4ac7d94de5e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:01.158515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d10eec-645b-4505-b69a-4ac7d94de5e8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:01.160114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fba94d43-2d70-4b26-805b-b86f3e909b17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6fd97062022478e9d34c318b309fd90'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:01.160739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fba94d43-2d70-4b26-805b-b86f3e909b17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6fd97062022478e9d34c318b309fd90'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:01.169401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:01.174793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1188/2408] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:00 2022] DELETE /identity/v3/users/b6fd97062022478e9d34c318b309fd90 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:01.271007 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1221/2409] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:01 2022] POST /identity/v3/projects => generated 405 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:24:01.275348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba94d43-2d70-4b26-805b-b86f3e909b17 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:01.337474 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba94d43-2d70-4b26-805b-b86f3e909b17 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:01.394145 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a88f36-5da0-4b7a-b2b9-036e1636f8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:01.394747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a88f36-5da0-4b7a-b2b9-036e1636f8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:01.395053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a88f36-5da0-4b7a-b2b9-036e1636f8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:01.395349 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a88f36-5da0-4b7a-b2b9-036e1636f8a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:01.396576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a88f36-5da0-4b7a-b2b9-036e1636f8a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:01.398596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a88f36-5da0-4b7a-b2b9-036e1636f8a8 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:01.480116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d10eec-645b-4505-b69a-4ac7d94de5e8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:01.481826 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-63d10eec-645b-4505-b69a-4ac7d94de5e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:01.492093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1189/2410] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:01 2022] POST /identity/v3/users => generated 303 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:01.497582 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d10eec-645b-4505-b69a-4ac7d94de5e8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:01.497830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a88f36-5da0-4b7a-b2b9-036e1636f8a8 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:01.505916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b950ab0-c80e-4c4e-b895-7534c0af8425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:01.506584 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b950ab0-c80e-4c4e-b895-7534c0af8425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:01.507122 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b950ab0-c80e-4c4e-b895-7534c0af8425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:01.507365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b950ab0-c80e-4c4e-b895-7534c0af8425 admin admin] PATH_INFO: `/v3/projects/f0458e73970e41b3ac7948f8eb8f75e9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:01.508749 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b950ab0-c80e-4c4e-b895-7534c0af8425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0458e73970e41b3ac7948f8eb8f75e9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:01.510617 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b950ab0-c80e-4c4e-b895-7534c0af8425 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:01.510733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a88f36-5da0-4b7a-b2b9-036e1636f8a8 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:01.516077 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b817b-4c72-427b-a12d-3489cccb3a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:01.516710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b817b-4c72-427b-a12d-3489cccb3a59 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:01.516988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b817b-4c72-427b-a12d-3489cccb3a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:01.517258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b817b-4c72-427b-a12d-3489cccb3a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:01.517906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b817b-4c72-427b-a12d-3489cccb3a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:01.519700 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b817b-4c72-427b-a12d-3489cccb3a59 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:01.531509 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1190/2411] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:24:01.535532 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b817b-4c72-427b-a12d-3489cccb3a59 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:01.545624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b817b-4c72-427b-a12d-3489cccb3a59 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:01.550256 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d11b39-851e-4c9b-a76b-2b4742131389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:01.550843 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d11b39-851e-4c9b-a76b-2b4742131389 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:01.551146 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d11b39-851e-4c9b-a76b-2b4742131389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:01.551419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d11b39-851e-4c9b-a76b-2b4742131389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:01.552420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d11b39-851e-4c9b-a76b-2b4742131389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:01.553990 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d11b39-851e-4c9b-a76b-2b4742131389 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:01.565453 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1191/2412] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:24:01.570255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d11b39-851e-4c9b-a76b-2b4742131389 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:01.579541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d11b39-851e-4c9b-a76b-2b4742131389 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:01.585027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88746e57-08af-4c66-b0f5-bd9f7c28aaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:01.585661 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88746e57-08af-4c66-b0f5-bd9f7c28aaf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:01.585939 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88746e57-08af-4c66-b0f5-bd9f7c28aaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:01.586209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88746e57-08af-4c66-b0f5-bd9f7c28aaf1 admin admin] PATH_INFO: `/v3/projects/b702053b661d462fa13591f7120c0a32/users/0ee126ca76dd4fe281c6e2c59fa71ae5/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:01.600591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88746e57-08af-4c66-b0f5-bd9f7c28aaf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b702053b661d462fa13591f7120c0a32, user_id=0ee126ca76dd4fe281c6e2c59fa71ae5, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:01.603395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88746e57-08af-4c66-b0f5-bd9f7c28aaf1 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:01.673319 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1192/2413] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:01 2022] PUT /identity/v3/projects/b702053b661d462fa13591f7120c0a32/users/0ee126ca76dd4fe281c6e2c59fa71ae5/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:24:01.679180 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf208e9-1900-4396-b099-6da7bb9f63cb None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:01.679571 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf208e9-1900-4396-b099-6da7bb9f63cb None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:01.679848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf208e9-1900-4396-b099-6da7bb9f63cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:01.685192 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2b950ab0-c80e-4c4e-b895-7534c0af8425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0458e73970e41b3ac7948f8eb8f75e9', 'request_id': 'req-2b950ab0-c80e-4c4e-b895-7534c0af8425'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:01.691389 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2b950ab0-c80e-4c4e-b895-7534c0af8425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0458e73970e41b3ac7948f8eb8f75e9', 'request_id': 'req-2b950ab0-c80e-4c4e-b895-7534c0af8425'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:01.700212 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-eaf208e9-1900-4396-b099-6da7bb9f63cb None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:01.701419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-eaf208e9-1900-4396-b099-6da7bb9f63cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:01.761405 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1193/2414] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:24:01.771930 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf208e9-1900-4396-b099-6da7bb9f63cb None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:01.773323 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-eaf208e9-1900-4396-b099-6da7bb9f63cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:01.777824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2b950ab0-c80e-4c4e-b895-7534c0af8425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0458e73970e41b3ac7948f8eb8f75e9', 'request_id': 'req-2b950ab0-c80e-4c4e-b895-7534c0af8425'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:01.790516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2b950ab0-c80e-4c4e-b895-7534c0af8425 admin admin] The token cache is being invalidate because project f0458e73970e41b3ac7948f8eb8f75e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:01.790891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2b950ab0-c80e-4c4e-b895-7534c0af8425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:01.791666 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2b950ab0-c80e-4c4e-b895-7534c0af8425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:01.793296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1222/2415] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:01 2022] DELETE /identity/v3/projects/f0458e73970e41b3ac7948f8eb8f75e9 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:24:01.797886 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e912d1-b393-4a21-864a-b7c2de5468fc None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:01.798171 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e912d1-b393-4a21-864a-b7c2de5468fc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:01.798446 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e912d1-b393-4a21-864a-b7c2de5468fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:01.820140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d7e912d1-b393-4a21-864a-b7c2de5468fc None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:01.823856 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d7e912d1-b393-4a21-864a-b7c2de5468fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:01.830542 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1223/2416] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:24:01.839160 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf208e9-1900-4396-b099-6da7bb9f63cb None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:01.841864 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4c553-ee60-460c-86e9-d4dec2013d72 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:01.841864 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4c553-ee60-460c-86e9-d4dec2013d72 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:01.841864 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4c553-ee60-460c-86e9-d4dec2013d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:01.842997 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eccfc0-5f39-45d7-815f-45587ee43b08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:01.843643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eccfc0-5f39-45d7-815f-45587ee43b08 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:01.843920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eccfc0-5f39-45d7-815f-45587ee43b08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:01.844182 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eccfc0-5f39-45d7-815f-45587ee43b08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:01.846329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-68eccfc0-5f39-45d7-815f-45587ee43b08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:01.856008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:01.862418 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-16d4c553-ee60-460c-86e9-d4dec2013d72 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:01.863659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-16d4c553-ee60-460c-86e9-d4dec2013d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:01.865159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eccfc0-5f39-45d7-815f-45587ee43b08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:01.866639 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eccfc0-5f39-45d7-815f-45587ee43b08 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:01.872640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1224/2417] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:24:01.876580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4c553-ee60-460c-86e9-d4dec2013d72 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:01.877836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1194/2418] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:24:01.886616 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4c553-ee60-460c-86e9-d4dec2013d72 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:01.891656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9949ed-50a1-4873-a5ea-e6a38a8ccd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:01.892500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9949ed-50a1-4873-a5ea-e6a38a8ccd63 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:01.892782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9949ed-50a1-4873-a5ea-e6a38a8ccd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:01.893058 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9949ed-50a1-4873-a5ea-e6a38a8ccd63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:01.893711 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9949ed-50a1-4873-a5ea-e6a38a8ccd63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:24:01.894013 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9949ed-50a1-4873-a5ea-e6a38a8ccd63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:01.896087 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9949ed-50a1-4873-a5ea-e6a38a8ccd63 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:01.907784 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1225/2419] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 08:24:01.912542 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eccfc0-5f39-45d7-815f-45587ee43b08 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:01.922151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eccfc0-5f39-45d7-815f-45587ee43b08 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:01.927509 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9af9db-6560-483b-a3f2-93ce48879117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:01.928177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9af9db-6560-483b-a3f2-93ce48879117 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:01.928457 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9af9db-6560-483b-a3f2-93ce48879117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:01.928727 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9af9db-6560-483b-a3f2-93ce48879117 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:01.929352 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9af9db-6560-483b-a3f2-93ce48879117 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:01.931299 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9af9db-6560-483b-a3f2-93ce48879117 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:02.050662 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1195/2420] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:01 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:02.052924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9949ed-50a1-4873-a5ea-e6a38a8ccd63 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:02.062986 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9949ed-50a1-4873-a5ea-e6a38a8ccd63 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:02.122766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cfb9e-00a5-4d8b-a8b2-dd98fbdf4f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:02.123392 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cfb9e-00a5-4d8b-a8b2-dd98fbdf4f9d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:02.123666 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cfb9e-00a5-4d8b-a8b2-dd98fbdf4f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:02.123933 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cfb9e-00a5-4d8b-a8b2-dd98fbdf4f9d admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:02.124588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cfb9e-00a5-4d8b-a8b2-dd98fbdf4f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:02.126498 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cfb9e-00a5-4d8b-a8b2-dd98fbdf4f9d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:02.183522 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:02.209074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1226/2421] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:02 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:02.210690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9af9db-6560-483b-a3f2-93ce48879117 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:02.221089 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9af9db-6560-483b-a3f2-93ce48879117 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:02.226406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeac645-5537-4b93-98b9-a52314c2dae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:02.227040 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeac645-5537-4b93-98b9-a52314c2dae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:02.229856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeac645-5537-4b93-98b9-a52314c2dae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:02.229856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeac645-5537-4b93-98b9-a52314c2dae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:02.229856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeac645-5537-4b93-98b9-a52314c2dae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:02.229856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeac645-5537-4b93-98b9-a52314c2dae4 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:02.240841 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1196/2422] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:02.245889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cfb9e-00a5-4d8b-a8b2-dd98fbdf4f9d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:02.254565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cfb9e-00a5-4d8b-a8b2-dd98fbdf4f9d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:02.259437 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f914bb4-62ba-4b2a-af74-aca1a47830e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:02.260024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f914bb4-62ba-4b2a-af74-aca1a47830e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:02.260303 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f914bb4-62ba-4b2a-af74-aca1a47830e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:02.260575 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f914bb4-62ba-4b2a-af74-aca1a47830e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:02.261210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f914bb4-62ba-4b2a-af74-aca1a47830e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:02.262806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f914bb4-62ba-4b2a-af74-aca1a47830e2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:02.273051 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1227/2423] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:02.277163 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeac645-5537-4b93-98b9-a52314c2dae4 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:02.289137 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeac645-5537-4b93-98b9-a52314c2dae4 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:02.293952 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18ca84-9975-43fc-9b43-92ec2f212d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:02.294692 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18ca84-9975-43fc-9b43-92ec2f212d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:02.295063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18ca84-9975-43fc-9b43-92ec2f212d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:02.295397 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18ca84-9975-43fc-9b43-92ec2f212d1f admin admin] PATH_INFO: `/v3/projects/d7fc0e02f9684319af6d220a40fdcafb/users/0b3e382ed3694c7d86b7022f130a58c1/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:02.310819 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18ca84-9975-43fc-9b43-92ec2f212d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7fc0e02f9684319af6d220a40fdcafb, user_id=0b3e382ed3694c7d86b7022f130a58c1, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:02.313604 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18ca84-9975-43fc-9b43-92ec2f212d1f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:02.447128 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1197/2424] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:02 2022] PUT /identity/v3/projects/d7fc0e02f9684319af6d220a40fdcafb/users/0b3e382ed3694c7d86b7022f130a58c1/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:02.457551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd169847-96de-4780-8bba-072dd4e6e375 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:02.458313 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd169847-96de-4780-8bba-072dd4e6e375 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:02.458748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd169847-96de-4780-8bba-072dd4e6e375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:02.486583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dd169847-96de-4780-8bba-072dd4e6e375 None None] MFA Rules not processed for user `0b3e382ed3694c7d86b7022f130a58c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:02.487823 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dd169847-96de-4780-8bba-072dd4e6e375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:02.552951 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1228/2425] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:02.563619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18ca84-9975-43fc-9b43-92ec2f212d1f admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:02.565427 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9b18ca84-9975-43fc-9b43-92ec2f212d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:02.622885 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18ca84-9975-43fc-9b43-92ec2f212d1f admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:02.626524 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe5599-f7ac-49d4-84a2-e7b6cf8eb507 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:02.627248 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe5599-f7ac-49d4-84a2-e7b6cf8eb507 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:02.627581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe5599-f7ac-49d4-84a2-e7b6cf8eb507 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:02.633406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe5599-f7ac-49d4-84a2-e7b6cf8eb507 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:02.639350 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe5599-f7ac-49d4-84a2-e7b6cf8eb507 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:02.640806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe5599-f7ac-49d4-84a2-e7b6cf8eb507 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:02.650379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1198/2426] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:03.177308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed4177-5c30-4e46-9524-490db32eae28 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:03.177845 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed4177-5c30-4e46-9524-490db32eae28 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:03.180203 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed4177-5c30-4e46-9524-490db32eae28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:03.224940 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3bed4177-5c30-4e46-9524-490db32eae28 None None] MFA Rules not processed for user `0ee126ca76dd4fe281c6e2c59fa71ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:03.226240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3bed4177-5c30-4e46-9524-490db32eae28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:03.274877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:03.297010 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1229/2427] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:03 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:03.302210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe5599-f7ac-49d4-84a2-e7b6cf8eb507 service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:03.303706 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9ffe5599-f7ac-49d4-84a2-e7b6cf8eb507 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:03.346034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:03.369241 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe5599-f7ac-49d4-84a2-e7b6cf8eb507 service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:03.373659 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0327d3f-19f2-4b09-91ce-4843ebcf12f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:03.374251 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327d3f-19f2-4b09-91ce-4843ebcf12f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:03.374525 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327d3f-19f2-4b09-91ce-4843ebcf12f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:03.374792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327d3f-19f2-4b09-91ce-4843ebcf12f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:03.375456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0327d3f-19f2-4b09-91ce-4843ebcf12f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:03.377419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0327d3f-19f2-4b09-91ce-4843ebcf12f8 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:03.561717 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1199/2428] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:03 2022] POST /identity/v3/projects => generated 405 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:24:03.563462 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed4177-5c30-4e46-9524-490db32eae28 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:03.574180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed4177-5c30-4e46-9524-490db32eae28 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:03.632200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f0fc4-89eb-4ce4-aac8-aed5b0041442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:03.632717 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f0fc4-89eb-4ce4-aac8-aed5b0041442 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:03.632985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f0fc4-89eb-4ce4-aac8-aed5b0041442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:03.633247 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f0fc4-89eb-4ce4-aac8-aed5b0041442 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:03.633888 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f0fc4-89eb-4ce4-aac8-aed5b0041442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:03.635846 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f0fc4-89eb-4ce4-aac8-aed5b0041442 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:03.727412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1230/2429] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:03 2022] POST /identity/v3/users => generated 309 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:24:03.731382 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0327d3f-19f2-4b09-91ce-4843ebcf12f8 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:03.742262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0327d3f-19f2-4b09-91ce-4843ebcf12f8 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:03.747449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ac2b9-9697-4bf6-aab4-ae7934bbf330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:03.748041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ac2b9-9697-4bf6-aab4-ae7934bbf330 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:03.748332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ac2b9-9697-4bf6-aab4-ae7934bbf330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:03.748580 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ac2b9-9697-4bf6-aab4-ae7934bbf330 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:03.749215 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ac2b9-9697-4bf6-aab4-ae7934bbf330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:03.750832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ac2b9-9697-4bf6-aab4-ae7934bbf330 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:03.761521 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1200/2430] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:03.773751 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f0fc4-89eb-4ce4-aac8-aed5b0041442 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:03.784020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f0fc4-89eb-4ce4-aac8-aed5b0041442 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:03.789794 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6571d-edff-46df-aedf-e81f883e51eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:03.790462 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6571d-edff-46df-aedf-e81f883e51eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:03.790720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6571d-edff-46df-aedf-e81f883e51eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:03.791015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6571d-edff-46df-aedf-e81f883e51eb admin admin] PATH_INFO: `/v3/projects/2999beefaf71472890de9c0cedeac44d/users/4b79ca65c3ae4937947ebd64abcabad6/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:03.806844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6571d-edff-46df-aedf-e81f883e51eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2999beefaf71472890de9c0cedeac44d, user_id=4b79ca65c3ae4937947ebd64abcabad6, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:03.809554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6571d-edff-46df-aedf-e81f883e51eb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:03.915411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1231/2431] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:03 2022] PUT /identity/v3/projects/2999beefaf71472890de9c0cedeac44d/users/4b79ca65c3ae4937947ebd64abcabad6/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:03.919242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ac2b9-9697-4bf6-aab4-ae7934bbf330 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:03.929421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ac2b9-9697-4bf6-aab4-ae7934bbf330 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:03.984243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5a6ea-d3e1-457c-b4cf-1168972542ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:03.984831 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5a6ea-d3e1-457c-b4cf-1168972542ec admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:03.985104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5a6ea-d3e1-457c-b4cf-1168972542ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:03.985371 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5a6ea-d3e1-457c-b4cf-1168972542ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:03.986002 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5a6ea-d3e1-457c-b4cf-1168972542ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:03.987633 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5a6ea-d3e1-457c-b4cf-1168972542ec admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:03.997061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1201/2432] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:04.001224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6571d-edff-46df-aedf-e81f883e51eb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:04.011984 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6571d-edff-46df-aedf-e81f883e51eb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:04.016895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ddf21-6f7d-4b48-8f80-7b4b7207d1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:04.017526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ddf21-6f7d-4b48-8f80-7b4b7207d1ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:04.017797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ddf21-6f7d-4b48-8f80-7b4b7207d1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:04.018066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ddf21-6f7d-4b48-8f80-7b4b7207d1ac admin admin] PATH_INFO: `/v3/projects/2999beefaf71472890de9c0cedeac44d/users/4b79ca65c3ae4937947ebd64abcabad6/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:04.019972 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ddf21-6f7d-4b48-8f80-7b4b7207d1ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=2999beefaf71472890de9c0cedeac44d, user_id=4b79ca65c3ae4937947ebd64abcabad6, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:04.023391 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ddf21-6f7d-4b48-8f80-7b4b7207d1ac admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:04.040597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1232/2433] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:04 2022] PUT /identity/v3/projects/2999beefaf71472890de9c0cedeac44d/users/4b79ca65c3ae4937947ebd64abcabad6/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:05.410725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00d005-f079-4df2-8f95-d364c65485f7 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:05.412151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00d005-f079-4df2-8f95-d364c65485f7 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:05.412582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00d005-f079-4df2-8f95-d364c65485f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:05.452558 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-0d00d005-f079-4df2-8f95-d364c65485f7 None None] MFA Rules not processed for user `4b79ca65c3ae4937947ebd64abcabad6`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:05.453946 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0d00d005-f079-4df2-8f95-d364c65485f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:05.534225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1202/2434] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:05 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:07.180044 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ddf21-6f7d-4b48-8f80-7b4b7207d1ac admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:07.247986 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ddf21-6f7d-4b48-8f80-7b4b7207d1ac admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:07.252810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babbc7f-6fb3-4b3d-bf95-46b3c488db8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:07.253424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babbc7f-6fb3-4b3d-bf95-46b3c488db8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:07.253695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babbc7f-6fb3-4b3d-bf95-46b3c488db8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:07.253958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babbc7f-6fb3-4b3d-bf95-46b3c488db8f admin admin] PATH_INFO: `/v3/users/0ee126ca76dd4fe281c6e2c59fa71ae5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:07.255673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8babbc7f-6fb3-4b3d-bf95-46b3c488db8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ee126ca76dd4fe281c6e2c59fa71ae5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:07.257587 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8babbc7f-6fb3-4b3d-bf95-46b3c488db8f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:07.460315 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:07.466625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8babbc7f-6fb3-4b3d-bf95-46b3c488db8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ee126ca76dd4fe281c6e2c59fa71ae5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:07.478534 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8babbc7f-6fb3-4b3d-bf95-46b3c488db8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ee126ca76dd4fe281c6e2c59fa71ae5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:07.479242 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8babbc7f-6fb3-4b3d-bf95-46b3c488db8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ee126ca76dd4fe281c6e2c59fa71ae5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:07.578225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8babbc7f-6fb3-4b3d-bf95-46b3c488db8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ee126ca76dd4fe281c6e2c59fa71ae5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:07.579152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8babbc7f-6fb3-4b3d-bf95-46b3c488db8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ee126ca76dd4fe281c6e2c59fa71ae5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:07.595488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1233/2435] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:07 2022] DELETE /identity/v3/users/0ee126ca76dd4fe281c6e2c59fa71ae5 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:07.871153 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00d005-f079-4df2-8f95-d364c65485f7 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:07.882380 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00d005-f079-4df2-8f95-d364c65485f7 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:07.916678 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:07.943247 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c264a-af8e-450b-abd6-0dbea34e8935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:07.943899 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c264a-af8e-450b-abd6-0dbea34e8935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:07.944478 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c264a-af8e-450b-abd6-0dbea34e8935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:07.944478 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c264a-af8e-450b-abd6-0dbea34e8935 admin admin] PATH_INFO: `/v3/projects/b702053b661d462fa13591f7120c0a32` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:07.945546 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c264a-af8e-450b-abd6-0dbea34e8935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b702053b661d462fa13591f7120c0a32)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:07.947518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c264a-af8e-450b-abd6-0dbea34e8935 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:08.118686 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-626c264a-af8e-450b-abd6-0dbea34e8935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b702053b661d462fa13591f7120c0a32', 'request_id': 'req-626c264a-af8e-450b-abd6-0dbea34e8935'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:08.124709 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-626c264a-af8e-450b-abd6-0dbea34e8935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b702053b661d462fa13591f7120c0a32', 'request_id': 'req-626c264a-af8e-450b-abd6-0dbea34e8935'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:08.206167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-626c264a-af8e-450b-abd6-0dbea34e8935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b702053b661d462fa13591f7120c0a32', 'request_id': 'req-626c264a-af8e-450b-abd6-0dbea34e8935'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:08.219357 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-626c264a-af8e-450b-abd6-0dbea34e8935 admin admin] The token cache is being invalidate because project b702053b661d462fa13591f7120c0a32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:08.219761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-626c264a-af8e-450b-abd6-0dbea34e8935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:08.220492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-626c264a-af8e-450b-abd6-0dbea34e8935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:08.222016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1203/2436] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:07 2022] DELETE /identity/v3/projects/b702053b661d462fa13591f7120c0a32 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:24:08.226035 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babbc7f-6fb3-4b3d-bf95-46b3c488db8f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:08.227548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8babbc7f-6fb3-4b3d-bf95-46b3c488db8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:08.240769 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babbc7f-6fb3-4b3d-bf95-46b3c488db8f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:08.299017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51a3b0-674c-46a4-ae21-caf66223279c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:08.299742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51a3b0-674c-46a4-ae21-caf66223279c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:08.300011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51a3b0-674c-46a4-ae21-caf66223279c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:08.300407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51a3b0-674c-46a4-ae21-caf66223279c admin admin] PATH_INFO: `/v3/users/4b79ca65c3ae4937947ebd64abcabad6` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:08.301635 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af51a3b0-674c-46a4-ae21-caf66223279c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b79ca65c3ae4937947ebd64abcabad6)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:08.303737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af51a3b0-674c-46a4-ae21-caf66223279c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:08.573107 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-af51a3b0-674c-46a4-ae21-caf66223279c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b79ca65c3ae4937947ebd64abcabad6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:08.585425 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-af51a3b0-674c-46a4-ae21-caf66223279c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b79ca65c3ae4937947ebd64abcabad6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:08.586106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-af51a3b0-674c-46a4-ae21-caf66223279c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b79ca65c3ae4937947ebd64abcabad6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:08.657368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-af51a3b0-674c-46a4-ae21-caf66223279c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b79ca65c3ae4937947ebd64abcabad6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:08.658338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-af51a3b0-674c-46a4-ae21-caf66223279c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b79ca65c3ae4937947ebd64abcabad6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:08.671581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1234/2437] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:08 2022] DELETE /identity/v3/users/4b79ca65c3ae4937947ebd64abcabad6 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:08.940156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c264a-af8e-450b-abd6-0dbea34e8935 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:08.941627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-626c264a-af8e-450b-abd6-0dbea34e8935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:08.967033 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c264a-af8e-450b-abd6-0dbea34e8935 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:09.039104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076002e-2a00-490a-b594-3c506bfacffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:09.039736 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076002e-2a00-490a-b594-3c506bfacffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:09.040015 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076002e-2a00-490a-b594-3c506bfacffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:09.040287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076002e-2a00-490a-b594-3c506bfacffd admin admin] PATH_INFO: `/v3/projects/2999beefaf71472890de9c0cedeac44d` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:09.041409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076002e-2a00-490a-b594-3c506bfacffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2999beefaf71472890de9c0cedeac44d)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:09.043360 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076002e-2a00-490a-b594-3c506bfacffd admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:09.203178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6076002e-2a00-490a-b594-3c506bfacffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2999beefaf71472890de9c0cedeac44d', 'request_id': 'req-6076002e-2a00-490a-b594-3c506bfacffd'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:09.209498 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6076002e-2a00-490a-b594-3c506bfacffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2999beefaf71472890de9c0cedeac44d', 'request_id': 'req-6076002e-2a00-490a-b594-3c506bfacffd'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:09.297352 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6076002e-2a00-490a-b594-3c506bfacffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2999beefaf71472890de9c0cedeac44d', 'request_id': 'req-6076002e-2a00-490a-b594-3c506bfacffd'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:09.312582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:09.312582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6076002e-2a00-490a-b594-3c506bfacffd admin admin] The token cache is being invalidate because project 2999beefaf71472890de9c0cedeac44d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:09.312582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6076002e-2a00-490a-b594-3c506bfacffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:09.312582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6076002e-2a00-490a-b594-3c506bfacffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:09.312582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1204/2438] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:08 2022] DELETE /identity/v3/projects/2999beefaf71472890de9c0cedeac44d => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:24:09.317878 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbc238-944a-4661-9c29-6de528ad23aa None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:09.319527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbc238-944a-4661-9c29-6de528ad23aa None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:09.319527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbc238-944a-4661-9c29-6de528ad23aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:09.339158 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4ebbc238-944a-4661-9c29-6de528ad23aa None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:09.340361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4ebbc238-944a-4661-9c29-6de528ad23aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:09.370968 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:09.400675 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1235/2439] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:09.408365 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67ed49-9d67-4076-909b-bfa2b35b062e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:09.408645 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67ed49-9d67-4076-909b-bfa2b35b062e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:09.408919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67ed49-9d67-4076-909b-bfa2b35b062e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:09.429709 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-ac67ed49-9d67-4076-909b-bfa2b35b062e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:09.430922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-ac67ed49-9d67-4076-909b-bfa2b35b062e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:09.439060 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1205/2440] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:09.444163 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbc238-944a-4661-9c29-6de528ad23aa None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:09.454544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbc238-944a-4661-9c29-6de528ad23aa None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:09.474236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aef8b2-7604-4d6e-9200-5c81924596d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:09.474236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aef8b2-7604-4d6e-9200-5c81924596d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:09.474236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aef8b2-7604-4d6e-9200-5c81924596d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:09.474236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aef8b2-7604-4d6e-9200-5c81924596d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:09.474625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aef8b2-7604-4d6e-9200-5c81924596d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:24:09.474625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aef8b2-7604-4d6e-9200-5c81924596d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:09.474625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aef8b2-7604-4d6e-9200-5c81924596d1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:09.480008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1236/2441] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 08:24:09.481192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67ed49-9d67-4076-909b-bfa2b35b062e None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:09.491002 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67ed49-9d67-4076-909b-bfa2b35b062e None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:09.497476 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e8c63-6d5b-4eab-9bdc-1d1eccdcb376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:09.499312 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e8c63-6d5b-4eab-9bdc-1d1eccdcb376 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:09.499312 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e8c63-6d5b-4eab-9bdc-1d1eccdcb376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:09.499312 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e8c63-6d5b-4eab-9bdc-1d1eccdcb376 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:09.499312 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e8c63-6d5b-4eab-9bdc-1d1eccdcb376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:09.501151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e8c63-6d5b-4eab-9bdc-1d1eccdcb376 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:09.612150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1206/2442] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:09 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:09.616376 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aef8b2-7604-4d6e-9200-5c81924596d1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:09.626359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aef8b2-7604-4d6e-9200-5c81924596d1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:09.686375 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd707d4d-7729-4f97-af66-615a891a083d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:09.686883 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd707d4d-7729-4f97-af66-615a891a083d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:09.687195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd707d4d-7729-4f97-af66-615a891a083d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:09.687458 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd707d4d-7729-4f97-af66-615a891a083d admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:09.688077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd707d4d-7729-4f97-af66-615a891a083d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:09.690006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd707d4d-7729-4f97-af66-615a891a083d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:09.776953 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1237/2443] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:09 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:09.781156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e8c63-6d5b-4eab-9bdc-1d1eccdcb376 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:09.791911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e8c63-6d5b-4eab-9bdc-1d1eccdcb376 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:09.796829 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252f090-1ceb-483b-9577-55af76e73cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:09.797502 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252f090-1ceb-483b-9577-55af76e73cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:09.797894 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252f090-1ceb-483b-9577-55af76e73cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:09.798232 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252f090-1ceb-483b-9577-55af76e73cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:09.798958 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252f090-1ceb-483b-9577-55af76e73cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:09.800791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252f090-1ceb-483b-9577-55af76e73cfc admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:09.811553 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1207/2444] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:09.815495 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd707d4d-7729-4f97-af66-615a891a083d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:09.827095 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd707d4d-7729-4f97-af66-615a891a083d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:09.831770 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75375ee9-aec9-4e5f-aa85-c1bea61809cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:09.832364 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75375ee9-aec9-4e5f-aa85-c1bea61809cc admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:09.833295 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75375ee9-aec9-4e5f-aa85-c1bea61809cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:09.834166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75375ee9-aec9-4e5f-aa85-c1bea61809cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:09.834815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75375ee9-aec9-4e5f-aa85-c1bea61809cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:09.836436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75375ee9-aec9-4e5f-aa85-c1bea61809cc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:09.846671 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1238/2445] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:09.850831 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252f090-1ceb-483b-9577-55af76e73cfc admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:09.861238 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252f090-1ceb-483b-9577-55af76e73cfc admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:09.866521 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b69be57-0541-4173-abc6-172bf075758b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:09.867200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69be57-0541-4173-abc6-172bf075758b admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:09.867479 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69be57-0541-4173-abc6-172bf075758b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:09.867754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b69be57-0541-4173-abc6-172bf075758b admin admin] PATH_INFO: `/v3/projects/c0842ce2997c4cb7b36b1acc3bb7216a/users/ab937f14a0914ef6a82f1aeaa0b292fc/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:09.883103 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69be57-0541-4173-abc6-172bf075758b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0842ce2997c4cb7b36b1acc3bb7216a, user_id=ab937f14a0914ef6a82f1aeaa0b292fc, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:09.885777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b69be57-0541-4173-abc6-172bf075758b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:09.953666 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75375ee9-aec9-4e5f-aa85-c1bea61809cc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:09.955410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-75375ee9-aec9-4e5f-aa85-c1bea61809cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:09.970612 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1208/2446] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:09 2022] PUT /identity/v3/projects/c0842ce2997c4cb7b36b1acc3bb7216a/users/ab937f14a0914ef6a82f1aeaa0b292fc/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:09.972161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef7acb-5c01-4c72-b0eb-4daddfcbcdd9 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:09.972376 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef7acb-5c01-4c72-b0eb-4daddfcbcdd9 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:09.972644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef7acb-5c01-4c72-b0eb-4daddfcbcdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:10.015480 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-edef7acb-5c01-4c72-b0eb-4daddfcbcdd9 None None] MFA Rules not processed for user `ab937f14a0914ef6a82f1aeaa0b292fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:10.016688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-edef7acb-5c01-4c72-b0eb-4daddfcbcdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:10.025247 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75375ee9-aec9-4e5f-aa85-c1bea61809cc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:10.087435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85500a4-552c-4dd4-bbef-b24ce725b5cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:10.088148 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85500a4-552c-4dd4-bbef-b24ce725b5cb service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:10.088352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85500a4-552c-4dd4-bbef-b24ce725b5cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:10.088599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85500a4-552c-4dd4-bbef-b24ce725b5cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:10.089851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1209/2447] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:10.093377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f85500a4-552c-4dd4-bbef-b24ce725b5cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:10.094654 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef7acb-5c01-4c72-b0eb-4daddfcbcdd9 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:10.100063 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:10.172808 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef7acb-5c01-4c72-b0eb-4daddfcbcdd9 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:10.178716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea8d1a-39f6-4465-ba4e-fdbc622df87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:10.179351 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea8d1a-39f6-4465-ba4e-fdbc622df87a admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:10.179755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea8d1a-39f6-4465-ba4e-fdbc622df87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:10.180034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea8d1a-39f6-4465-ba4e-fdbc622df87a admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:10.180661 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea8d1a-39f6-4465-ba4e-fdbc622df87a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:10.182579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea8d1a-39f6-4465-ba4e-fdbc622df87a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:10.192784 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85500a4-552c-4dd4-bbef-b24ce725b5cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:10.194614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85500a4-552c-4dd4-bbef-b24ce725b5cb service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:10.214775 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1239/2448] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:09 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:24:10.293402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1210/2449] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:10 2022] POST /identity/v3/projects => generated 379 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:10.294841 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85500a4-552c-4dd4-bbef-b24ce725b5cb service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:10.310040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85500a4-552c-4dd4-bbef-b24ce725b5cb service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:10.382069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e166e-0980-4009-adb6-9ca2bb4bfc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:10.382679 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e166e-0980-4009-adb6-9ca2bb4bfc3e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:10.382932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e166e-0980-4009-adb6-9ca2bb4bfc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:10.383225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e166e-0980-4009-adb6-9ca2bb4bfc3e admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:10.393660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e166e-0980-4009-adb6-9ca2bb4bfc3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:10.393660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e166e-0980-4009-adb6-9ca2bb4bfc3e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:10.465365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1240/2450] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:10 2022] POST /identity/v3/users => generated 296 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:24:10.470298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea8d1a-39f6-4465-ba4e-fdbc622df87a admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:10.487158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:10.487158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea8d1a-39f6-4465-ba4e-fdbc622df87a admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:10.487158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da6d71-5eac-4c7b-bcec-a3d20b41bcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:10.487158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da6d71-5eac-4c7b-bcec-a3d20b41bcd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:10.487532 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da6d71-5eac-4c7b-bcec-a3d20b41bcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:10.487532 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da6d71-5eac-4c7b-bcec-a3d20b41bcd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:10.487982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da6d71-5eac-4c7b-bcec-a3d20b41bcd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:10.489915 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da6d71-5eac-4c7b-bcec-a3d20b41bcd9 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:10.502966 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1211/2451] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:10.506323 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e166e-0980-4009-adb6-9ca2bb4bfc3e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:10.516862 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e166e-0980-4009-adb6-9ca2bb4bfc3e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:10.522129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5a921-d1dd-4b9e-95ef-abdde8c34fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:10.522903 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5a921-d1dd-4b9e-95ef-abdde8c34fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:10.523202 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5a921-d1dd-4b9e-95ef-abdde8c34fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:10.523473 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5a921-d1dd-4b9e-95ef-abdde8c34fb8 admin admin] PATH_INFO: `/v3/projects/477aad49cbd24ef998864901201a8b40/users/04cdd6683d304a67889c3cf844c2f69c/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:10.539400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5a921-d1dd-4b9e-95ef-abdde8c34fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=477aad49cbd24ef998864901201a8b40, user_id=04cdd6683d304a67889c3cf844c2f69c, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:10.542134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5a921-d1dd-4b9e-95ef-abdde8c34fb8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:10.594328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1241/2452] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:10 2022] PUT /identity/v3/projects/477aad49cbd24ef998864901201a8b40/users/04cdd6683d304a67889c3cf844c2f69c/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:10.598606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da6d71-5eac-4c7b-bcec-a3d20b41bcd9 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:10.609406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da6d71-5eac-4c7b-bcec-a3d20b41bcd9 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:10.667992 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00669f51-f788-4c0d-80f4-9b12f2d0f40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:10.668598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00669f51-f788-4c0d-80f4-9b12f2d0f40c admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:10.668871 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00669f51-f788-4c0d-80f4-9b12f2d0f40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:10.669153 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00669f51-f788-4c0d-80f4-9b12f2d0f40c admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:10.669751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00669f51-f788-4c0d-80f4-9b12f2d0f40c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:10.671423 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00669f51-f788-4c0d-80f4-9b12f2d0f40c admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:10.682359 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1212/2453] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:24:10.686430 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5a921-d1dd-4b9e-95ef-abdde8c34fb8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:10.712665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5a921-d1dd-4b9e-95ef-abdde8c34fb8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:10.718460 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36f290-0498-44a0-96ab-4db2196edfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:10.719312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36f290-0498-44a0-96ab-4db2196edfbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:10.719645 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36f290-0498-44a0-96ab-4db2196edfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:10.722292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36f290-0498-44a0-96ab-4db2196edfbf admin admin] PATH_INFO: `/v3/projects/477aad49cbd24ef998864901201a8b40/users/04cdd6683d304a67889c3cf844c2f69c/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:10.722292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36f290-0498-44a0-96ab-4db2196edfbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=477aad49cbd24ef998864901201a8b40, user_id=04cdd6683d304a67889c3cf844c2f69c, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:10.726016 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36f290-0498-44a0-96ab-4db2196edfbf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:10.747576 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1242/2454] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:10 2022] PUT /identity/v3/projects/477aad49cbd24ef998864901201a8b40/users/04cdd6683d304a67889c3cf844c2f69c/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:10.753095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13561bb-faf9-4bf6-b820-ce63f2747aaf None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:10.753638 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13561bb-faf9-4bf6-b820-ce63f2747aaf None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:10.754324 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13561bb-faf9-4bf6-b820-ce63f2747aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:10.803182 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-d13561bb-faf9-4bf6-b820-ce63f2747aaf None None] MFA Rules not processed for user `04cdd6683d304a67889c3cf844c2f69c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:10.803182 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d13561bb-faf9-4bf6-b820-ce63f2747aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:10.891853 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1213/2455] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:10.905066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36f290-0498-44a0-96ab-4db2196edfbf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:10.906765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5e36f290-0498-44a0-96ab-4db2196edfbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:10.954266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:10.976066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36f290-0498-44a0-96ab-4db2196edfbf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:10.979776 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ae043-c543-457e-bb4c-ed07ec2a6402 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:10.980373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ae043-c543-457e-bb4c-ed07ec2a6402 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:10.980644 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ae043-c543-457e-bb4c-ed07ec2a6402 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:10.980912 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ae043-c543-457e-bb4c-ed07ec2a6402 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:10.990362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ae043-c543-457e-bb4c-ed07ec2a6402 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:10.991830 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ae043-c543-457e-bb4c-ed07ec2a6402 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:11.001805 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1243/2456] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:10 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:11.865474 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13561bb-faf9-4bf6-b820-ce63f2747aaf None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:11.874733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13561bb-faf9-4bf6-b820-ce63f2747aaf None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:11.879979 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a687acb-5b18-41c4-a5ec-0f5f2e4a5380 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cdd6683d304a67889c3cf844c2f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477aad49cbd24ef998864901201a8b40', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:11.881903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a687acb-5b18-41c4-a5ec-0f5f2e4a5380 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:11.881903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a687acb-5b18-41c4-a5ec-0f5f2e4a5380 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:11.881903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a687acb-5b18-41c4-a5ec-0f5f2e4a5380 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:11.882308 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a687acb-5b18-41c4-a5ec-0f5f2e4a5380 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:24:11.882616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a687acb-5b18-41c4-a5ec-0f5f2e4a5380 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:11.884260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a687acb-5b18-41c4-a5ec-0f5f2e4a5380 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:11.897454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1214/2457] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:24:11.901639 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ae043-c543-457e-bb4c-ed07ec2a6402 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:11.910649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ae043-c543-457e-bb4c-ed07ec2a6402 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:11.915324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50466249-9b67-4004-a546-c2ccb788028f tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cdd6683d304a67889c3cf844c2f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477aad49cbd24ef998864901201a8b40', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:11.915908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50466249-9b67-4004-a546-c2ccb788028f tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:11.916183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50466249-9b67-4004-a546-c2ccb788028f tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:11.916449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50466249-9b67-4004-a546-c2ccb788028f tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:11.917078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50466249-9b67-4004-a546-c2ccb788028f tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:11.919004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50466249-9b67-4004-a546-c2ccb788028f tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:12.006466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1244/2458] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:11 2022] POST /identity/v3/projects => generated 354 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:24:12.010667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a687acb-5b18-41c4-a5ec-0f5f2e4a5380 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:12.021736 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a687acb-5b18-41c4-a5ec-0f5f2e4a5380 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:12.091367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2508b-97b5-4ae5-b130-206b7377b288 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cdd6683d304a67889c3cf844c2f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477aad49cbd24ef998864901201a8b40', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:12.092599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2508b-97b5-4ae5-b130-206b7377b288 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:12.092599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2508b-97b5-4ae5-b130-206b7377b288 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:12.093083 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2508b-97b5-4ae5-b130-206b7377b288 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:12.093823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2508b-97b5-4ae5-b130-206b7377b288 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:24:12.094218 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2508b-97b5-4ae5-b130-206b7377b288 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:12.095953 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2508b-97b5-4ae5-b130-206b7377b288 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:12.099599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:12.109583 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1215/2459] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:24:12.149259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50466249-9b67-4004-a546-c2ccb788028f tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:12.161484 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50466249-9b67-4004-a546-c2ccb788028f tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:12.167900 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57188b41-d05a-422e-9e6e-de3840a63d24 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cdd6683d304a67889c3cf844c2f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '477aad49cbd24ef998864901201a8b40', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:12.168271 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57188b41-d05a-422e-9e6e-de3840a63d24 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:12.168934 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57188b41-d05a-422e-9e6e-de3840a63d24 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:12.168934 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57188b41-d05a-422e-9e6e-de3840a63d24 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] PATH_INFO: `/v3/projects/51e7e39954794361ac1e3056d30e7d08` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:12.170009 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57188b41-d05a-422e-9e6e-de3840a63d24 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] RBAC: Authorizing `identity:delete_project(project_id=51e7e39954794361ac1e3056d30e7d08)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:12.171998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57188b41-d05a-422e-9e6e-de3840a63d24 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:12.297634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-57188b41-d05a-422e-9e6e-de3840a63d24 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51e7e39954794361ac1e3056d30e7d08', 'request_id': 'req-57188b41-d05a-422e-9e6e-de3840a63d24'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:12.304551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-57188b41-d05a-422e-9e6e-de3840a63d24 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51e7e39954794361ac1e3056d30e7d08', 'request_id': 'req-57188b41-d05a-422e-9e6e-de3840a63d24'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:12.469824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-57188b41-d05a-422e-9e6e-de3840a63d24 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51e7e39954794361ac1e3056d30e7d08', 'request_id': 'req-57188b41-d05a-422e-9e6e-de3840a63d24'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:12.481932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-57188b41-d05a-422e-9e6e-de3840a63d24 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] The token cache is being invalidate because project 51e7e39954794361ac1e3056d30e7d08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:12.482220 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-57188b41-d05a-422e-9e6e-de3840a63d24 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:12.482890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-57188b41-d05a-422e-9e6e-de3840a63d24 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:12.484667 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1245/2460] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:12 2022] DELETE /identity/v3/projects/51e7e39954794361ac1e3056d30e7d08 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:13.038365 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2508b-97b5-4ae5-b130-206b7377b288 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:13.040168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-28c2508b-97b5-4ae5-b130-206b7377b288 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:13.110161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2508b-97b5-4ae5-b130-206b7377b288 tempest-VolumeQuotaClassesTest-914801311 tempest-VolumeQuotaClassesTest-914801311-project-admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:13.187926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8196b-6594-4447-9bd5-bc96e2f48c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:13.189069 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8196b-6594-4447-9bd5-bc96e2f48c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:13.189558 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8196b-6594-4447-9bd5-bc96e2f48c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:13.190036 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8196b-6594-4447-9bd5-bc96e2f48c71 admin admin] PATH_INFO: `/v3/users/ab937f14a0914ef6a82f1aeaa0b292fc` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:13.191495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8196b-6594-4447-9bd5-bc96e2f48c71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab937f14a0914ef6a82f1aeaa0b292fc)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:13.195086 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8196b-6594-4447-9bd5-bc96e2f48c71 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:13.419264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6fb8196b-6594-4447-9bd5-bc96e2f48c71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab937f14a0914ef6a82f1aeaa0b292fc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:13.419933 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6fb8196b-6594-4447-9bd5-bc96e2f48c71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab937f14a0914ef6a82f1aeaa0b292fc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:13.439888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6fb8196b-6594-4447-9bd5-bc96e2f48c71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab937f14a0914ef6a82f1aeaa0b292fc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:13.508739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6fb8196b-6594-4447-9bd5-bc96e2f48c71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab937f14a0914ef6a82f1aeaa0b292fc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:13.509429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6fb8196b-6594-4447-9bd5-bc96e2f48c71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab937f14a0914ef6a82f1aeaa0b292fc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:13.523834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1216/2461] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:13 2022] DELETE /identity/v3/users/ab937f14a0914ef6a82f1aeaa0b292fc => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:13.528732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657eaba-8d7f-4a1f-a39c-283e91aff3e2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:13.529249 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657eaba-8d7f-4a1f-a39c-283e91aff3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:13.529249 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657eaba-8d7f-4a1f-a39c-283e91aff3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:13.549840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b657eaba-8d7f-4a1f-a39c-283e91aff3e2 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:13.551038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b657eaba-8d7f-4a1f-a39c-283e91aff3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:13.617976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1246/2462] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:13.628497 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8196b-6594-4447-9bd5-bc96e2f48c71 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:13.630243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6fb8196b-6594-4447-9bd5-bc96e2f48c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:13.658590 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:13.692440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8196b-6594-4447-9bd5-bc96e2f48c71 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:13.696735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30978b53-fded-4cc2-ba92-be22f5222f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:13.697340 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30978b53-fded-4cc2-ba92-be22f5222f40 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:13.697641 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30978b53-fded-4cc2-ba92-be22f5222f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:13.697910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30978b53-fded-4cc2-ba92-be22f5222f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:13.709250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30978b53-fded-4cc2-ba92-be22f5222f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:13.710796 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30978b53-fded-4cc2-ba92-be22f5222f40 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:13.720753 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1217/2463] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:14.215674 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657eaba-8d7f-4a1f-a39c-283e91aff3e2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:14.217690 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b657eaba-8d7f-4a1f-a39c-283e91aff3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:14.232877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657eaba-8d7f-4a1f-a39c-283e91aff3e2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:14.237729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764407bf-d991-45a5-98ab-703dd4095fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:14.243365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764407bf-d991-45a5-98ab-703dd4095fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:14.243365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764407bf-d991-45a5-98ab-703dd4095fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:14.243365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764407bf-d991-45a5-98ab-703dd4095fd5 admin admin] PATH_INFO: `/v3/projects/c0842ce2997c4cb7b36b1acc3bb7216a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:14.243365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764407bf-d991-45a5-98ab-703dd4095fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0842ce2997c4cb7b36b1acc3bb7216a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:14.243365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764407bf-d991-45a5-98ab-703dd4095fd5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:14.359894 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:14.382541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-764407bf-d991-45a5-98ab-703dd4095fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0842ce2997c4cb7b36b1acc3bb7216a', 'request_id': 'req-764407bf-d991-45a5-98ab-703dd4095fd5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:14.389965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-764407bf-d991-45a5-98ab-703dd4095fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0842ce2997c4cb7b36b1acc3bb7216a', 'request_id': 'req-764407bf-d991-45a5-98ab-703dd4095fd5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:14.497968 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-764407bf-d991-45a5-98ab-703dd4095fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0842ce2997c4cb7b36b1acc3bb7216a', 'request_id': 'req-764407bf-d991-45a5-98ab-703dd4095fd5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:14.509749 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-764407bf-d991-45a5-98ab-703dd4095fd5 admin admin] The token cache is being invalidate because project c0842ce2997c4cb7b36b1acc3bb7216a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:14.510121 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-764407bf-d991-45a5-98ab-703dd4095fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:14.510737 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-764407bf-d991-45a5-98ab-703dd4095fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:14.512239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1247/2464] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:14 2022] DELETE /identity/v3/projects/c0842ce2997c4cb7b36b1acc3bb7216a => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:14.516600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30978b53-fded-4cc2-ba92-be22f5222f40 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:14.526328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30978b53-fded-4cc2-ba92-be22f5222f40 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:14.584179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae8b95-ba89-49bc-a970-da4ed30a0596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:14.584815 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae8b95-ba89-49bc-a970-da4ed30a0596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:14.585099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae8b95-ba89-49bc-a970-da4ed30a0596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:14.585368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae8b95-ba89-49bc-a970-da4ed30a0596 admin admin] PATH_INFO: `/v3/users/04cdd6683d304a67889c3cf844c2f69c` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:14.586873 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae8b95-ba89-49bc-a970-da4ed30a0596 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04cdd6683d304a67889c3cf844c2f69c)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:14.592174 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae8b95-ba89-49bc-a970-da4ed30a0596 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:14.829392 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b8ae8b95-ba89-49bc-a970-da4ed30a0596 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04cdd6683d304a67889c3cf844c2f69c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:14.830076 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b8ae8b95-ba89-49bc-a970-da4ed30a0596 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04cdd6683d304a67889c3cf844c2f69c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:14.842891 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b8ae8b95-ba89-49bc-a970-da4ed30a0596 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04cdd6683d304a67889c3cf844c2f69c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:14.925966 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b8ae8b95-ba89-49bc-a970-da4ed30a0596 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04cdd6683d304a67889c3cf844c2f69c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:14.926739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b8ae8b95-ba89-49bc-a970-da4ed30a0596 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04cdd6683d304a67889c3cf844c2f69c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:14.939734 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1218/2465] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:14 2022] DELETE /identity/v3/users/04cdd6683d304a67889c3cf844c2f69c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:24:15.366349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764407bf-d991-45a5-98ab-703dd4095fd5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:15.367882 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-764407bf-d991-45a5-98ab-703dd4095fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:15.382512 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764407bf-d991-45a5-98ab-703dd4095fd5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:15.441514 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398b6b5-dffc-41fd-bddf-b64795ed709c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:15.442020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398b6b5-dffc-41fd-bddf-b64795ed709c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:15.442293 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398b6b5-dffc-41fd-bddf-b64795ed709c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:15.443278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398b6b5-dffc-41fd-bddf-b64795ed709c admin admin] PATH_INFO: `/v3/projects/477aad49cbd24ef998864901201a8b40` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:15.444613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398b6b5-dffc-41fd-bddf-b64795ed709c admin admin] RBAC: Authorizing `identity:delete_project(project_id=477aad49cbd24ef998864901201a8b40)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:15.446544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398b6b5-dffc-41fd-bddf-b64795ed709c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:15.532922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae8b95-ba89-49bc-a970-da4ed30a0596 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:15.534364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b8ae8b95-ba89-49bc-a970-da4ed30a0596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:15.548294 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae8b95-ba89-49bc-a970-da4ed30a0596 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:15.549851 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d398b6b5-dffc-41fd-bddf-b64795ed709c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '477aad49cbd24ef998864901201a8b40', 'request_id': 'req-d398b6b5-dffc-41fd-bddf-b64795ed709c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:15.557298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d398b6b5-dffc-41fd-bddf-b64795ed709c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '477aad49cbd24ef998864901201a8b40', 'request_id': 'req-d398b6b5-dffc-41fd-bddf-b64795ed709c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:15.593507 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:15.609126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66065719-24e5-4b1a-85ec-182e4fc8f0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:15.609678 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66065719-24e5-4b1a-85ec-182e4fc8f0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:15.609953 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66065719-24e5-4b1a-85ec-182e4fc8f0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:15.610223 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66065719-24e5-4b1a-85ec-182e4fc8f0ba admin admin] PATH_INFO: `/v3/users/0b3e382ed3694c7d86b7022f130a58c1` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:15.611395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66065719-24e5-4b1a-85ec-182e4fc8f0ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b3e382ed3694c7d86b7022f130a58c1)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:15.613340 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66065719-24e5-4b1a-85ec-182e4fc8f0ba admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:15.640398 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d398b6b5-dffc-41fd-bddf-b64795ed709c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '477aad49cbd24ef998864901201a8b40', 'request_id': 'req-d398b6b5-dffc-41fd-bddf-b64795ed709c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:15.656818 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d398b6b5-dffc-41fd-bddf-b64795ed709c admin admin] The token cache is being invalidate because project 477aad49cbd24ef998864901201a8b40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:15.657256 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d398b6b5-dffc-41fd-bddf-b64795ed709c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:15.657941 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d398b6b5-dffc-41fd-bddf-b64795ed709c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:15.659507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1248/2466] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:15 2022] DELETE /identity/v3/projects/477aad49cbd24ef998864901201a8b40 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:15.666242 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44889dec-a048-4245-b1b4-fd3310b4e2fe None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:15.666523 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44889dec-a048-4245-b1b4-fd3310b4e2fe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:15.666795 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44889dec-a048-4245-b1b4-fd3310b4e2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:15.688636 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-44889dec-a048-4245-b1b4-fd3310b4e2fe None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:15.689848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-44889dec-a048-4245-b1b4-fd3310b4e2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:15.698618 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1249/2467] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:15.706021 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5d2be-4ef0-4dee-959e-5ea87440859f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:15.706300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5d2be-4ef0-4dee-959e-5ea87440859f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:15.706565 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5d2be-4ef0-4dee-959e-5ea87440859f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:15.727422 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dfa5d2be-4ef0-4dee-959e-5ea87440859f None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:15.728619 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dfa5d2be-4ef0-4dee-959e-5ea87440859f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:15.736644 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1250/2468] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:24:15.742048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5d2be-4ef0-4dee-959e-5ea87440859f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:15.750961 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5d2be-4ef0-4dee-959e-5ea87440859f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:15.756258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0984272-9d47-4549-a93b-8a079b550b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:15.756828 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0984272-9d47-4549-a93b-8a079b550b95 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:15.757102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0984272-9d47-4549-a93b-8a079b550b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:15.757370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0984272-9d47-4549-a93b-8a079b550b95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:15.758030 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0984272-9d47-4549-a93b-8a079b550b95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:24:15.758324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0984272-9d47-4549-a93b-8a079b550b95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:15.759950 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0984272-9d47-4549-a93b-8a079b550b95 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:15.772162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1251/2469] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 08:24:15.776401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0984272-9d47-4549-a93b-8a079b550b95 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:15.786105 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0984272-9d47-4549-a93b-8a079b550b95 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:15.791422 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fdbaf-67c0-4005-b3dc-637b7fb4c146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:15.791985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fdbaf-67c0-4005-b3dc-637b7fb4c146 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:15.792265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fdbaf-67c0-4005-b3dc-637b7fb4c146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:15.792540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fdbaf-67c0-4005-b3dc-637b7fb4c146 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:15.793151 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591fdbaf-67c0-4005-b3dc-637b7fb4c146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:15.795042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591fdbaf-67c0-4005-b3dc-637b7fb4c146 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:15.846644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-66065719-24e5-4b1a-85ec-182e4fc8f0ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b3e382ed3694c7d86b7022f130a58c1'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:15.847424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-66065719-24e5-4b1a-85ec-182e4fc8f0ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b3e382ed3694c7d86b7022f130a58c1'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:15.868858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-66065719-24e5-4b1a-85ec-182e4fc8f0ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b3e382ed3694c7d86b7022f130a58c1'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:15.894292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1252/2470] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:15 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:15.899706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fdbaf-67c0-4005-b3dc-637b7fb4c146 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:15.902366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-591fdbaf-67c0-4005-b3dc-637b7fb4c146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:15.908503 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:15.938481 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-66065719-24e5-4b1a-85ec-182e4fc8f0ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b3e382ed3694c7d86b7022f130a58c1'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:15.938481 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-66065719-24e5-4b1a-85ec-182e4fc8f0ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b3e382ed3694c7d86b7022f130a58c1'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:15.955668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1219/2471] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:15 2022] DELETE /identity/v3/users/0b3e382ed3694c7d86b7022f130a58c1 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:24:15.960618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76e521-f9c3-48e5-8dfa-ce6bb5ba6b30 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:15.960893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76e521-f9c3-48e5-8dfa-ce6bb5ba6b30 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:15.961167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76e521-f9c3-48e5-8dfa-ce6bb5ba6b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:15.975165 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fdbaf-67c0-4005-b3dc-637b7fb4c146 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:15.982251 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-0b76e521-f9c3-48e5-8dfa-ce6bb5ba6b30 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:15.983493 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0b76e521-f9c3-48e5-8dfa-ce6bb5ba6b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:16.042109 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f83d6-01f0-4a5f-b36b-3bcb23bc1e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:16.042841 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f83d6-01f0-4a5f-b36b-3bcb23bc1e5c admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:16.043273 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f83d6-01f0-4a5f-b36b-3bcb23bc1e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:16.043626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f83d6-01f0-4a5f-b36b-3bcb23bc1e5c admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:16.044269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f83d6-01f0-4a5f-b36b-3bcb23bc1e5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:16.046548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f83d6-01f0-4a5f-b36b-3bcb23bc1e5c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:16.049503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1220/2472] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:24:16.060492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76e521-f9c3-48e5-8dfa-ce6bb5ba6b30 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:16.061915 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0b76e521-f9c3-48e5-8dfa-ce6bb5ba6b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:16.125660 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76e521-f9c3-48e5-8dfa-ce6bb5ba6b30 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:16.129290 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e19dee-2844-4633-b73e-c32878872382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:16.129922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e19dee-2844-4633-b73e-c32878872382 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:16.130275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e19dee-2844-4633-b73e-c32878872382 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:16.130545 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e19dee-2844-4633-b73e-c32878872382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:16.143389 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e19dee-2844-4633-b73e-c32878872382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:16.146078 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e19dee-2844-4633-b73e-c32878872382 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:16.153734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1253/2473] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:15 2022] POST /identity/v3/users => generated 294 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:24:16.154740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f83d6-01f0-4a5f-b36b-3bcb23bc1e5c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:16.160409 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1221/2474] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:16.166331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f83d6-01f0-4a5f-b36b-3bcb23bc1e5c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:16.172556 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6793e40-54b2-4019-8c55-05382d1e7864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:16.173366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6793e40-54b2-4019-8c55-05382d1e7864 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:16.174385 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6793e40-54b2-4019-8c55-05382d1e7864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:16.174385 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6793e40-54b2-4019-8c55-05382d1e7864 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:16.174805 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6793e40-54b2-4019-8c55-05382d1e7864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:16.176810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6793e40-54b2-4019-8c55-05382d1e7864 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:16.187515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1254/2475] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:16.191615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e19dee-2844-4633-b73e-c32878872382 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:16.201919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e19dee-2844-4633-b73e-c32878872382 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:16.207064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c230bc7-1136-4784-8a3a-8a1913870e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:16.207700 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c230bc7-1136-4784-8a3a-8a1913870e81 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:16.208546 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c230bc7-1136-4784-8a3a-8a1913870e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:16.208806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c230bc7-1136-4784-8a3a-8a1913870e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:16.209482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c230bc7-1136-4784-8a3a-8a1913870e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:16.211237 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c230bc7-1136-4784-8a3a-8a1913870e81 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:16.212626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:16.222207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1222/2476] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:16.227139 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6793e40-54b2-4019-8c55-05382d1e7864 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:16.237268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6793e40-54b2-4019-8c55-05382d1e7864 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:16.242540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546e158-2763-41ce-841d-96ee86b1f5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:16.243268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546e158-2763-41ce-841d-96ee86b1f5bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:16.243693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546e158-2763-41ce-841d-96ee86b1f5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:16.243969 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546e158-2763-41ce-841d-96ee86b1f5bc admin admin] PATH_INFO: `/v3/projects/46a7f8be51934c6eb7e65d8be6ec1aac/users/5f367c7304894956a451d60c635ed0de/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:16.260699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546e158-2763-41ce-841d-96ee86b1f5bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a7f8be51934c6eb7e65d8be6ec1aac, user_id=5f367c7304894956a451d60c635ed0de, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:16.263541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546e158-2763-41ce-841d-96ee86b1f5bc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:16.346420 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1255/2477] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:16 2022] PUT /identity/v3/projects/46a7f8be51934c6eb7e65d8be6ec1aac/users/5f367c7304894956a451d60c635ed0de/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:24:16.356870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5403ec-029c-4774-b89d-281fe115891b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:16.357107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5403ec-029c-4774-b89d-281fe115891b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:16.357441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5403ec-029c-4774-b89d-281fe115891b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:16.410742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-9d5403ec-029c-4774-b89d-281fe115891b None None] MFA Rules not processed for user `5f367c7304894956a451d60c635ed0de`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:16.412385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9d5403ec-029c-4774-b89d-281fe115891b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:16.479916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1223/2478] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:16.484548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546e158-2763-41ce-841d-96ee86b1f5bc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:16.548860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546e158-2763-41ce-841d-96ee86b1f5bc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:16.553963 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d527e72-423f-4eee-9536-1967e07db70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:16.554554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d527e72-423f-4eee-9536-1967e07db70d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:16.554824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d527e72-423f-4eee-9536-1967e07db70d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:16.555128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d527e72-423f-4eee-9536-1967e07db70d admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:16.555768 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d527e72-423f-4eee-9536-1967e07db70d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:16.557710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d527e72-423f-4eee-9536-1967e07db70d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:16.569437 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:16.611525 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5403ec-029c-4774-b89d-281fe115891b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:16.613046 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9d5403ec-029c-4774-b89d-281fe115891b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:16.629799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5403ec-029c-4774-b89d-281fe115891b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:16.640460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15076f81-2fb2-4d0c-a4ec-e8334554e850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:16.640460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15076f81-2fb2-4d0c-a4ec-e8334554e850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:16.640460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15076f81-2fb2-4d0c-a4ec-e8334554e850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:16.640759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15076f81-2fb2-4d0c-a4ec-e8334554e850 admin admin] PATH_INFO: `/v3/projects/d7fc0e02f9684319af6d220a40fdcafb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:16.641626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15076f81-2fb2-4d0c-a4ec-e8334554e850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7fc0e02f9684319af6d220a40fdcafb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:16.643605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15076f81-2fb2-4d0c-a4ec-e8334554e850 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:16.656378 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1256/2479] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:16 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:16.662054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d527e72-423f-4eee-9536-1967e07db70d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:16.728142 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d527e72-423f-4eee-9536-1967e07db70d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:16.732798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c2857-7621-40ca-8297-d743c5a84fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:16.733389 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c2857-7621-40ca-8297-d743c5a84fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:16.733843 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c2857-7621-40ca-8297-d743c5a84fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:16.734117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c2857-7621-40ca-8297-d743c5a84fb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:16.734734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c2857-7621-40ca-8297-d743c5a84fb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:16.736682 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c2857-7621-40ca-8297-d743c5a84fb4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:16.752044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-15076f81-2fb2-4d0c-a4ec-e8334554e850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7fc0e02f9684319af6d220a40fdcafb', 'request_id': 'req-15076f81-2fb2-4d0c-a4ec-e8334554e850'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:16.758798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-15076f81-2fb2-4d0c-a4ec-e8334554e850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7fc0e02f9684319af6d220a40fdcafb', 'request_id': 'req-15076f81-2fb2-4d0c-a4ec-e8334554e850'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:16.806316 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1257/2480] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:16 2022] POST /identity/v3/users => generated 300 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:24:16.810428 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c2857-7621-40ca-8297-d743c5a84fb4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:16.820184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-15076f81-2fb2-4d0c-a4ec-e8334554e850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7fc0e02f9684319af6d220a40fdcafb', 'request_id': 'req-15076f81-2fb2-4d0c-a4ec-e8334554e850'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:16.822515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c2857-7621-40ca-8297-d743c5a84fb4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:16.836719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-15076f81-2fb2-4d0c-a4ec-e8334554e850 admin admin] The token cache is being invalidate because project d7fc0e02f9684319af6d220a40fdcafb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:16.837186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-15076f81-2fb2-4d0c-a4ec-e8334554e850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:16.838153 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-15076f81-2fb2-4d0c-a4ec-e8334554e850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:16.840609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1224/2481] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:16 2022] DELETE /identity/v3/projects/d7fc0e02f9684319af6d220a40fdcafb => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:24:16.891630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a977b2-fd0e-4aa5-b2ab-195e9cf3ed2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:16.892159 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a977b2-fd0e-4aa5-b2ab-195e9cf3ed2d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:16.892447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a977b2-fd0e-4aa5-b2ab-195e9cf3ed2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:16.892700 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a977b2-fd0e-4aa5-b2ab-195e9cf3ed2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:16.893357 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a977b2-fd0e-4aa5-b2ab-195e9cf3ed2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:16.895115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a977b2-fd0e-4aa5-b2ab-195e9cf3ed2d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:16.906763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1258/2482] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:24:16.911407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15076f81-2fb2-4d0c-a4ec-e8334554e850 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:16.912991 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-15076f81-2fb2-4d0c-a4ec-e8334554e850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:16.926387 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15076f81-2fb2-4d0c-a4ec-e8334554e850 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:16.932174 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070bf583-c8cb-4f7b-adb9-7462971a1f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:16.932857 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bf583-c8cb-4f7b-adb9-7462971a1f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:16.933133 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bf583-c8cb-4f7b-adb9-7462971a1f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:16.933411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bf583-c8cb-4f7b-adb9-7462971a1f4b admin admin] PATH_INFO: `/v3/projects/6048b02cc1cd404796b192d3a6c14ffd/users/d54149c3e4ba4eb9addfb3018c372696/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:16.950851 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070bf583-c8cb-4f7b-adb9-7462971a1f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6048b02cc1cd404796b192d3a6c14ffd, user_id=d54149c3e4ba4eb9addfb3018c372696, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:16.953980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070bf583-c8cb-4f7b-adb9-7462971a1f4b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:17.012461 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1225/2483] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:16 2022] PUT /identity/v3/projects/6048b02cc1cd404796b192d3a6c14ffd/users/d54149c3e4ba4eb9addfb3018c372696/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:17.017056 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a977b2-fd0e-4aa5-b2ab-195e9cf3ed2d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:17.027704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a977b2-fd0e-4aa5-b2ab-195e9cf3ed2d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:17.090430 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d388e-3551-49c1-8b73-80cde932c002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:17.091101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d388e-3551-49c1-8b73-80cde932c002 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:17.091378 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d388e-3551-49c1-8b73-80cde932c002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:17.091646 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d388e-3551-49c1-8b73-80cde932c002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:17.092295 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d388e-3551-49c1-8b73-80cde932c002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:17.093961 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d388e-3551-49c1-8b73-80cde932c002 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:17.105644 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1259/2484] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:17.109760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070bf583-c8cb-4f7b-adb9-7462971a1f4b admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:17.120872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070bf583-c8cb-4f7b-adb9-7462971a1f4b admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:17.125612 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea621517-ab73-4345-ba4b-be028abc04ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:17.126275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea621517-ab73-4345-ba4b-be028abc04ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:17.126568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea621517-ab73-4345-ba4b-be028abc04ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:17.127300 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea621517-ab73-4345-ba4b-be028abc04ca admin admin] PATH_INFO: `/v3/projects/6048b02cc1cd404796b192d3a6c14ffd/users/d54149c3e4ba4eb9addfb3018c372696/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:17.128808 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea621517-ab73-4345-ba4b-be028abc04ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=6048b02cc1cd404796b192d3a6c14ffd, user_id=d54149c3e4ba4eb9addfb3018c372696, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:17.131647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea621517-ab73-4345-ba4b-be028abc04ca admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:17.146369 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1226/2485] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:17 2022] PUT /identity/v3/projects/6048b02cc1cd404796b192d3a6c14ffd/users/d54149c3e4ba4eb9addfb3018c372696/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:24:17.154284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110a363-6d4f-4100-8af5-6f6a7dbf53b1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:17.154595 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110a363-6d4f-4100-8af5-6f6a7dbf53b1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:17.154827 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110a363-6d4f-4100-8af5-6f6a7dbf53b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:17.182214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:17.185086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3110a363-6d4f-4100-8af5-6f6a7dbf53b1 None None] MFA Rules not processed for user `d54149c3e4ba4eb9addfb3018c372696`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:17.186359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3110a363-6d4f-4100-8af5-6f6a7dbf53b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:17.252973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1260/2486] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:17 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:17.264161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea621517-ab73-4345-ba4b-be028abc04ca admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:17.265848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-ea621517-ab73-4345-ba4b-be028abc04ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:17.300688 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:17.327467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea621517-ab73-4345-ba4b-be028abc04ca admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:17.331181 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dfaa6-e931-43ff-8ae2-3f9fedd43dee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:17.331793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dfaa6-e931-43ff-8ae2-3f9fedd43dee service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:17.332068 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dfaa6-e931-43ff-8ae2-3f9fedd43dee service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:17.332335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dfaa6-e931-43ff-8ae2-3f9fedd43dee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:17.344772 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dfaa6-e931-43ff-8ae2-3f9fedd43dee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:17.346291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dfaa6-e931-43ff-8ae2-3f9fedd43dee service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:17.358086 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1227/2487] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:17.498146 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110a363-6d4f-4100-8af5-6f6a7dbf53b1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:17.559621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110a363-6d4f-4100-8af5-6f6a7dbf53b1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:17.564105 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ee62b-8e62-49d2-a758-6f9f1f24aaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:17.564722 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ee62b-8e62-49d2-a758-6f9f1f24aaf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:17.565002 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ee62b-8e62-49d2-a758-6f9f1f24aaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:17.565273 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ee62b-8e62-49d2-a758-6f9f1f24aaf4 admin admin] PATH_INFO: `/v3/users/5f367c7304894956a451d60c635ed0de` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:17.566379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ee62b-8e62-49d2-a758-6f9f1f24aaf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f367c7304894956a451d60c635ed0de)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:17.568528 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ee62b-8e62-49d2-a758-6f9f1f24aaf4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:17.766430 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-067ee62b-8e62-49d2-a758-6f9f1f24aaf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f367c7304894956a451d60c635ed0de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:17.779430 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-067ee62b-8e62-49d2-a758-6f9f1f24aaf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f367c7304894956a451d60c635ed0de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:17.780210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-067ee62b-8e62-49d2-a758-6f9f1f24aaf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f367c7304894956a451d60c635ed0de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:17.867020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-067ee62b-8e62-49d2-a758-6f9f1f24aaf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f367c7304894956a451d60c635ed0de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:17.867664 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-067ee62b-8e62-49d2-a758-6f9f1f24aaf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f367c7304894956a451d60c635ed0de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:17.881214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1261/2488] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:17 2022] DELETE /identity/v3/users/5f367c7304894956a451d60c635ed0de => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:17.885717 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441677df-a85e-4d25-b72f-63f49a85f7a4 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:17.886198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441677df-a85e-4d25-b72f-63f49a85f7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:17.886323 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441677df-a85e-4d25-b72f-63f49a85f7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:17.906294 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-441677df-a85e-4d25-b72f-63f49a85f7a4 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:17.907545 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-441677df-a85e-4d25-b72f-63f49a85f7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:17.965546 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1228/2489] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:17.975264 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ee62b-8e62-49d2-a758-6f9f1f24aaf4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:17.976587 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-067ee62b-8e62-49d2-a758-6f9f1f24aaf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:17.996577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:18.036278 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ee62b-8e62-49d2-a758-6f9f1f24aaf4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:18.040370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69514211-2698-4eb7-997f-085dedcf335e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:18.040998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69514211-2698-4eb7-997f-085dedcf335e service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:18.041226 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69514211-2698-4eb7-997f-085dedcf335e service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:18.041522 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69514211-2698-4eb7-997f-085dedcf335e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:18.051134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69514211-2698-4eb7-997f-085dedcf335e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:18.052715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69514211-2698-4eb7-997f-085dedcf335e service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:18.063985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1262/2490] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:18.511289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441677df-a85e-4d25-b72f-63f49a85f7a4 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:18.512677 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-441677df-a85e-4d25-b72f-63f49a85f7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:18.526427 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441677df-a85e-4d25-b72f-63f49a85f7a4 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:18.531114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6065702-8e4e-4335-91bd-57af8e7bc2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:18.531916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6065702-8e4e-4335-91bd-57af8e7bc2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:18.532049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6065702-8e4e-4335-91bd-57af8e7bc2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:18.532317 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6065702-8e4e-4335-91bd-57af8e7bc2a8 admin admin] PATH_INFO: `/v3/projects/46a7f8be51934c6eb7e65d8be6ec1aac` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:18.534432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6065702-8e4e-4335-91bd-57af8e7bc2a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46a7f8be51934c6eb7e65d8be6ec1aac)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:18.537229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6065702-8e4e-4335-91bd-57af8e7bc2a8 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:18.652347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b6065702-8e4e-4335-91bd-57af8e7bc2a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46a7f8be51934c6eb7e65d8be6ec1aac', 'request_id': 'req-b6065702-8e4e-4335-91bd-57af8e7bc2a8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:18.658606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b6065702-8e4e-4335-91bd-57af8e7bc2a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46a7f8be51934c6eb7e65d8be6ec1aac', 'request_id': 'req-b6065702-8e4e-4335-91bd-57af8e7bc2a8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:18.741481 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b6065702-8e4e-4335-91bd-57af8e7bc2a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46a7f8be51934c6eb7e65d8be6ec1aac', 'request_id': 'req-b6065702-8e4e-4335-91bd-57af8e7bc2a8'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:18.752477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b6065702-8e4e-4335-91bd-57af8e7bc2a8 admin admin] The token cache is being invalidate because project 46a7f8be51934c6eb7e65d8be6ec1aac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:18.752765 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b6065702-8e4e-4335-91bd-57af8e7bc2a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:18.753452 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b6065702-8e4e-4335-91bd-57af8e7bc2a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:18.755017 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1229/2491] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:18 2022] DELETE /identity/v3/projects/46a7f8be51934c6eb7e65d8be6ec1aac => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:18.759072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69514211-2698-4eb7-997f-085dedcf335e service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:18.768982 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69514211-2698-4eb7-997f-085dedcf335e service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:18.825979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61ce3d-8c42-4196-81cd-8db631c76d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:18.826597 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61ce3d-8c42-4196-81cd-8db631c76d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:18.826881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61ce3d-8c42-4196-81cd-8db631c76d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:18.827166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61ce3d-8c42-4196-81cd-8db631c76d4d admin admin] PATH_INFO: `/v3/users/d54149c3e4ba4eb9addfb3018c372696` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:18.828517 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61ce3d-8c42-4196-81cd-8db631c76d4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d54149c3e4ba4eb9addfb3018c372696)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:18.830435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61ce3d-8c42-4196-81cd-8db631c76d4d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:18.985482 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-aa61ce3d-8c42-4196-81cd-8db631c76d4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd54149c3e4ba4eb9addfb3018c372696'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:18.997028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-aa61ce3d-8c42-4196-81cd-8db631c76d4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd54149c3e4ba4eb9addfb3018c372696'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:18.997683 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-aa61ce3d-8c42-4196-81cd-8db631c76d4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd54149c3e4ba4eb9addfb3018c372696'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:19.079706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-aa61ce3d-8c42-4196-81cd-8db631c76d4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd54149c3e4ba4eb9addfb3018c372696'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:19.080223 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-aa61ce3d-8c42-4196-81cd-8db631c76d4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd54149c3e4ba4eb9addfb3018c372696'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:19.093827 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1263/2492] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:18 2022] DELETE /identity/v3/users/d54149c3e4ba4eb9addfb3018c372696 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:19.205449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6065702-8e4e-4335-91bd-57af8e7bc2a8 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:19.205449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b6065702-8e4e-4335-91bd-57af8e7bc2a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:19.259995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6065702-8e4e-4335-91bd-57af8e7bc2a8 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:19.268855 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:19.308685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1191ce-d8b6-4aff-a66d-55d33e3fee5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:19.309368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1191ce-d8b6-4aff-a66d-55d33e3fee5d service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:19.309646 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1191ce-d8b6-4aff-a66d-55d33e3fee5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:19.309914 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1191ce-d8b6-4aff-a66d-55d33e3fee5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:19.311685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-cf1191ce-d8b6-4aff-a66d-55d33e3fee5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:19.377153 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1191ce-d8b6-4aff-a66d-55d33e3fee5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:19.378604 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1191ce-d8b6-4aff-a66d-55d33e3fee5d service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:19.389698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1230/2493] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:19.928924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61ce3d-8c42-4196-81cd-8db631c76d4d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:19.930358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-aa61ce3d-8c42-4196-81cd-8db631c76d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:19.943257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61ce3d-8c42-4196-81cd-8db631c76d4d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:19.948103 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8100f-09b0-40ab-a514-94832535c340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:19.949171 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8100f-09b0-40ab-a514-94832535c340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:19.949689 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8100f-09b0-40ab-a514-94832535c340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:19.949979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8100f-09b0-40ab-a514-94832535c340 admin admin] PATH_INFO: `/v3/projects/6048b02cc1cd404796b192d3a6c14ffd` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:19.951206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8100f-09b0-40ab-a514-94832535c340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6048b02cc1cd404796b192d3a6c14ffd)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:19.953140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8100f-09b0-40ab-a514-94832535c340 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:20.081641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:20.097580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d8d8100f-09b0-40ab-a514-94832535c340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6048b02cc1cd404796b192d3a6c14ffd', 'request_id': 'req-d8d8100f-09b0-40ab-a514-94832535c340'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:20.107354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d8d8100f-09b0-40ab-a514-94832535c340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6048b02cc1cd404796b192d3a6c14ffd', 'request_id': 'req-d8d8100f-09b0-40ab-a514-94832535c340'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:20.198571 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d8d8100f-09b0-40ab-a514-94832535c340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6048b02cc1cd404796b192d3a6c14ffd', 'request_id': 'req-d8d8100f-09b0-40ab-a514-94832535c340'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:20.210160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d8d8100f-09b0-40ab-a514-94832535c340 admin admin] The token cache is being invalidate because project 6048b02cc1cd404796b192d3a6c14ffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:20.210446 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d8d8100f-09b0-40ab-a514-94832535c340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:20.211128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d8d8100f-09b0-40ab-a514-94832535c340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:20.212663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1264/2494] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:19 2022] DELETE /identity/v3/projects/6048b02cc1cd404796b192d3a6c14ffd => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:20.217689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc202731-be36-415d-89ef-82bd7f65405b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:20.218210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc202731-be36-415d-89ef-82bd7f65405b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:20.218457 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc202731-be36-415d-89ef-82bd7f65405b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:20.240833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-dc202731-be36-415d-89ef-82bd7f65405b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:20.242464 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-dc202731-be36-415d-89ef-82bd7f65405b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:20.313548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1231/2495] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:20.321836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74138c9b-ab0e-4d86-a7fa-79a0d15bb110 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:20.322120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74138c9b-ab0e-4d86-a7fa-79a0d15bb110 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:20.322392 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74138c9b-ab0e-4d86-a7fa-79a0d15bb110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:20.344265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-74138c9b-ab0e-4d86-a7fa-79a0d15bb110 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:20.345548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-74138c9b-ab0e-4d86-a7fa-79a0d15bb110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:20.354343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1265/2496] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:24:20.358864 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc202731-be36-415d-89ef-82bd7f65405b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:20.369578 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc202731-be36-415d-89ef-82bd7f65405b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:20.374234 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da554bd0-0217-4888-91e5-d22d4b7cb480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:20.374850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da554bd0-0217-4888-91e5-d22d4b7cb480 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:20.381935 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da554bd0-0217-4888-91e5-d22d4b7cb480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:20.381935 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da554bd0-0217-4888-91e5-d22d4b7cb480 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:20.381935 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da554bd0-0217-4888-91e5-d22d4b7cb480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:24:20.381935 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da554bd0-0217-4888-91e5-d22d4b7cb480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:20.381935 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da554bd0-0217-4888-91e5-d22d4b7cb480 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:20.390799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1232/2497] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:24:20.395218 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74138c9b-ab0e-4d86-a7fa-79a0d15bb110 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:20.404815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74138c9b-ab0e-4d86-a7fa-79a0d15bb110 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:20.409460 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6ff34-8497-40f9-9ad2-9277483a657c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:20.410342 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6ff34-8497-40f9-9ad2-9277483a657c admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:20.410617 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6ff34-8497-40f9-9ad2-9277483a657c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:20.410881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6ff34-8497-40f9-9ad2-9277483a657c admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:20.412368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6ff34-8497-40f9-9ad2-9277483a657c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:20.414230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6ff34-8497-40f9-9ad2-9277483a657c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:20.508450 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1266/2498] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:20 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:20.512690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da554bd0-0217-4888-91e5-d22d4b7cb480 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:20.523555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da554bd0-0217-4888-91e5-d22d4b7cb480 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:20.570358 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:20.580183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19245d8-9fe3-4deb-9651-39a284f11ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:20.580782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19245d8-9fe3-4deb-9651-39a284f11ddd admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:20.581084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19245d8-9fe3-4deb-9651-39a284f11ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:20.581373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19245d8-9fe3-4deb-9651-39a284f11ddd admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:20.582023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19245d8-9fe3-4deb-9651-39a284f11ddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:20.585068 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19245d8-9fe3-4deb-9651-39a284f11ddd admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:20.668181 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1233/2499] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:20 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:20.669527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6ff34-8497-40f9-9ad2-9277483a657c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:20.681314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6ff34-8497-40f9-9ad2-9277483a657c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:20.686457 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4770f-0306-49c4-98d4-23fa4dd2f6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:20.687165 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4770f-0306-49c4-98d4-23fa4dd2f6be admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:20.687562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4770f-0306-49c4-98d4-23fa4dd2f6be admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:20.688494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4770f-0306-49c4-98d4-23fa4dd2f6be admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:20.688664 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4770f-0306-49c4-98d4-23fa4dd2f6be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:20.690622 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4770f-0306-49c4-98d4-23fa4dd2f6be admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:20.702882 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1267/2500] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:20.706644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19245d8-9fe3-4deb-9651-39a284f11ddd admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:20.719598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19245d8-9fe3-4deb-9651-39a284f11ddd admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:20.726068 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a893cd7-66ac-4992-b6f4-b2a2d74ad925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:20.726673 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a893cd7-66ac-4992-b6f4-b2a2d74ad925 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:20.726973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a893cd7-66ac-4992-b6f4-b2a2d74ad925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:20.727254 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a893cd7-66ac-4992-b6f4-b2a2d74ad925 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:20.727900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a893cd7-66ac-4992-b6f4-b2a2d74ad925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:20.729526 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a893cd7-66ac-4992-b6f4-b2a2d74ad925 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:20.742173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1234/2501] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:20.746659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4770f-0306-49c4-98d4-23fa4dd2f6be admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:20.759707 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4770f-0306-49c4-98d4-23fa4dd2f6be admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:20.764915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d40f92-5b6a-4659-9a9e-52df3c1e8245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:20.767086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d40f92-5b6a-4659-9a9e-52df3c1e8245 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:20.767086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d40f92-5b6a-4659-9a9e-52df3c1e8245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:20.767086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d40f92-5b6a-4659-9a9e-52df3c1e8245 admin admin] PATH_INFO: `/v3/projects/90ebfd98a43043a19dd06ebd2c96cd17/users/ce62a1e880f047bf935bf31bf11c71ff/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:20.783744 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d40f92-5b6a-4659-9a9e-52df3c1e8245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90ebfd98a43043a19dd06ebd2c96cd17, user_id=ce62a1e880f047bf935bf31bf11c71ff, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:20.786658 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d40f92-5b6a-4659-9a9e-52df3c1e8245 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:20.862193 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1268/2502] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:20 2022] PUT /identity/v3/projects/90ebfd98a43043a19dd06ebd2c96cd17/users/ce62a1e880f047bf935bf31bf11c71ff/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:24:20.872866 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb95142-8c72-4483-9abb-20ea7397f3a7 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:20.876491 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb95142-8c72-4483-9abb-20ea7397f3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:20.876491 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb95142-8c72-4483-9abb-20ea7397f3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:20.910479 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-3bb95142-8c72-4483-9abb-20ea7397f3a7 None None] MFA Rules not processed for user `ce62a1e880f047bf935bf31bf11c71ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:20.912498 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3bb95142-8c72-4483-9abb-20ea7397f3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:20.977831 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1235/2503] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:20.982303 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d40f92-5b6a-4659-9a9e-52df3c1e8245 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:21.045261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d40f92-5b6a-4659-9a9e-52df3c1e8245 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:21.049781 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571cff3-e4ec-4069-910b-fe0ed1e72795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:21.050383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571cff3-e4ec-4069-910b-fe0ed1e72795 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:21.050662 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571cff3-e4ec-4069-910b-fe0ed1e72795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:21.050922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571cff3-e4ec-4069-910b-fe0ed1e72795 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:21.051653 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571cff3-e4ec-4069-910b-fe0ed1e72795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:21.054769 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571cff3-e4ec-4069-910b-fe0ed1e72795 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:21.166982 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1269/2504] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:20 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:21.170993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb95142-8c72-4483-9abb-20ea7397f3a7 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:21.182150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb95142-8c72-4483-9abb-20ea7397f3a7 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:21.249069 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00947a-3446-4e6f-863f-c91fe4b34609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:21.249773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00947a-3446-4e6f-863f-c91fe4b34609 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:21.250010 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00947a-3446-4e6f-863f-c91fe4b34609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:21.250314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00947a-3446-4e6f-863f-c91fe4b34609 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:21.251745 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00947a-3446-4e6f-863f-c91fe4b34609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:21.255228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00947a-3446-4e6f-863f-c91fe4b34609 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:21.385956 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1236/2505] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:21 2022] POST /identity/v3/users => generated 289 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:21.390305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571cff3-e4ec-4069-910b-fe0ed1e72795 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:21.401721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571cff3-e4ec-4069-910b-fe0ed1e72795 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:21.408266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b4fad-6e40-475c-935b-25630f0c4ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:21.408867 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b4fad-6e40-475c-935b-25630f0c4ade admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:21.409138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b4fad-6e40-475c-935b-25630f0c4ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:21.409401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b4fad-6e40-475c-935b-25630f0c4ade admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:21.410103 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b4fad-6e40-475c-935b-25630f0c4ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:21.412338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b4fad-6e40-475c-935b-25630f0c4ade admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:21.426492 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1270/2506] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:21.427789 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00947a-3446-4e6f-863f-c91fe4b34609 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:21.437889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00947a-3446-4e6f-863f-c91fe4b34609 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:21.442841 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415c956-a437-4353-9bd5-0df4b6e4d2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:21.443569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415c956-a437-4353-9bd5-0df4b6e4d2a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:21.443932 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415c956-a437-4353-9bd5-0df4b6e4d2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:21.444247 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415c956-a437-4353-9bd5-0df4b6e4d2a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:21.445007 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415c956-a437-4353-9bd5-0df4b6e4d2a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:21.446664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415c956-a437-4353-9bd5-0df4b6e4d2a9 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:21.457456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1237/2507] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:21.462568 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b4fad-6e40-475c-935b-25630f0c4ade admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:21.473021 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b4fad-6e40-475c-935b-25630f0c4ade admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:21.479645 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8adcc-eb25-4d10-87f5-4fb013eef1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:21.479921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8adcc-eb25-4d10-87f5-4fb013eef1ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:21.480200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8adcc-eb25-4d10-87f5-4fb013eef1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:21.480473 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8adcc-eb25-4d10-87f5-4fb013eef1ca admin admin] PATH_INFO: `/v3/projects/2b4fd4d2a3b64812b23999ece1c402c7/users/b7e6b462a2cc4bf5a867fbcac3131b85/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:21.498886 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8adcc-eb25-4d10-87f5-4fb013eef1ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b4fd4d2a3b64812b23999ece1c402c7, user_id=b7e6b462a2cc4bf5a867fbcac3131b85, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:21.501631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8adcc-eb25-4d10-87f5-4fb013eef1ca admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:21.602042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1271/2508] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:21 2022] PUT /identity/v3/projects/2b4fd4d2a3b64812b23999ece1c402c7/users/b7e6b462a2cc4bf5a867fbcac3131b85/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:21.614020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6c26f-82b8-4254-abbb-371fa4d600f0 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:21.614588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6c26f-82b8-4254-abbb-371fa4d600f0 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:21.615083 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6c26f-82b8-4254-abbb-371fa4d600f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:21.649858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-73b6c26f-82b8-4254-abbb-371fa4d600f0 None None] MFA Rules not processed for user `b7e6b462a2cc4bf5a867fbcac3131b85`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:21.651352 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-73b6c26f-82b8-4254-abbb-371fa4d600f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:21.682043 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:21.723012 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1238/2509] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:21.728303 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8adcc-eb25-4d10-87f5-4fb013eef1ca admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:21.743560 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:21.797178 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8adcc-eb25-4d10-87f5-4fb013eef1ca admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:21.803499 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6157df-4d50-4c3a-9916-a12e7b1ded12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:21.804100 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6157df-4d50-4c3a-9916-a12e7b1ded12 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:21.804375 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6157df-4d50-4c3a-9916-a12e7b1ded12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:21.804646 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6157df-4d50-4c3a-9916-a12e7b1ded12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:21.805279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6157df-4d50-4c3a-9916-a12e7b1ded12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:21.807246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6157df-4d50-4c3a-9916-a12e7b1ded12 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:21.903932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1272/2510] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:21 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:21.905184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6c26f-82b8-4254-abbb-371fa4d600f0 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:21.917494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6c26f-82b8-4254-abbb-371fa4d600f0 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:21.977867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048a2e8-ac2f-4fab-b986-e1de2889a6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:21.978488 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048a2e8-ac2f-4fab-b986-e1de2889a6f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:21.978764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048a2e8-ac2f-4fab-b986-e1de2889a6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:21.979064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048a2e8-ac2f-4fab-b986-e1de2889a6f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:21.979701 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048a2e8-ac2f-4fab-b986-e1de2889a6f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:21.981826 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048a2e8-ac2f-4fab-b986-e1de2889a6f9 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:22.087500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1239/2511] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:21 2022] POST /identity/v3/users => generated 295 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:22.088740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6157df-4d50-4c3a-9916-a12e7b1ded12 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:22.101337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6157df-4d50-4c3a-9916-a12e7b1ded12 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:22.108155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fa29a-f893-4473-9590-f084010e2f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:22.109034 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fa29a-f893-4473-9590-f084010e2f13 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:22.109034 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fa29a-f893-4473-9590-f084010e2f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:22.109742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fa29a-f893-4473-9590-f084010e2f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:22.109973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97fa29a-f893-4473-9590-f084010e2f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:22.111617 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97fa29a-f893-4473-9590-f084010e2f13 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:22.130870 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1273/2512] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:22.135090 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048a2e8-ac2f-4fab-b986-e1de2889a6f9 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:22.146594 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048a2e8-ac2f-4fab-b986-e1de2889a6f9 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:22.151234 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9982c55-9186-46ed-9698-ba5a95ff2f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:22.151886 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9982c55-9186-46ed-9698-ba5a95ff2f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:22.152202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9982c55-9186-46ed-9698-ba5a95ff2f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:22.152477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9982c55-9186-46ed-9698-ba5a95ff2f63 admin admin] PATH_INFO: `/v3/projects/81b276a06a44467695cd7d498ea27bc2/users/006b5eab31ef455ba8de0e1b311151c7/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:22.169433 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9982c55-9186-46ed-9698-ba5a95ff2f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81b276a06a44467695cd7d498ea27bc2, user_id=006b5eab31ef455ba8de0e1b311151c7, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:22.173383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9982c55-9186-46ed-9698-ba5a95ff2f63 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:22.249590 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1240/2513] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:22 2022] PUT /identity/v3/projects/81b276a06a44467695cd7d498ea27bc2/users/006b5eab31ef455ba8de0e1b311151c7/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:22.253581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fa29a-f893-4473-9590-f084010e2f13 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:22.264623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fa29a-f893-4473-9590-f084010e2f13 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:22.323071 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503180a-9ea2-4df3-ad0c-90de828df555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:22.323680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503180a-9ea2-4df3-ad0c-90de828df555 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:22.324001 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503180a-9ea2-4df3-ad0c-90de828df555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:22.324258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503180a-9ea2-4df3-ad0c-90de828df555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:22.324897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503180a-9ea2-4df3-ad0c-90de828df555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:22.326506 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503180a-9ea2-4df3-ad0c-90de828df555 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:22.340485 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1274/2514] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:22.341663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9982c55-9186-46ed-9698-ba5a95ff2f63 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:22.352394 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9982c55-9186-46ed-9698-ba5a95ff2f63 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:22.357166 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411218a-f0a9-4f85-90be-38df2d5ac19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:22.357801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411218a-f0a9-4f85-90be-38df2d5ac19a admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:22.358082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411218a-f0a9-4f85-90be-38df2d5ac19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:22.358357 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411218a-f0a9-4f85-90be-38df2d5ac19a admin admin] PATH_INFO: `/v3/projects/81b276a06a44467695cd7d498ea27bc2/users/006b5eab31ef455ba8de0e1b311151c7/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:22.360329 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411218a-f0a9-4f85-90be-38df2d5ac19a admin admin] RBAC: Authorizing `identity:create_grant(project_id=81b276a06a44467695cd7d498ea27bc2, user_id=006b5eab31ef455ba8de0e1b311151c7, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:22.363020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411218a-f0a9-4f85-90be-38df2d5ac19a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:22.373689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1241/2515] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:22 2022] PUT /identity/v3/projects/81b276a06a44467695cd7d498ea27bc2/users/006b5eab31ef455ba8de0e1b311151c7/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:22.385046 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc0b59-575b-4c36-88d5-2883b6d402a8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:22.385269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc0b59-575b-4c36-88d5-2883b6d402a8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:22.385544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc0b59-575b-4c36-88d5-2883b6d402a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:22.414621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-07dc0b59-575b-4c36-88d5-2883b6d402a8 None None] MFA Rules not processed for user `006b5eab31ef455ba8de0e1b311151c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:22.415869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-07dc0b59-575b-4c36-88d5-2883b6d402a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:22.492048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1275/2516] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:22.504439 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411218a-f0a9-4f85-90be-38df2d5ac19a admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:22.508862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7411218a-f0a9-4f85-90be-38df2d5ac19a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:22.594501 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411218a-f0a9-4f85-90be-38df2d5ac19a admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:22.598973 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d28645-988e-4a4a-9ba7-6dee618fa5bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:22.599519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d28645-988e-4a4a-9ba7-6dee618fa5bd service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:22.599802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d28645-988e-4a4a-9ba7-6dee618fa5bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:22.604162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d28645-988e-4a4a-9ba7-6dee618fa5bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:22.617310 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d28645-988e-4a4a-9ba7-6dee618fa5bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:22.618974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d28645-988e-4a4a-9ba7-6dee618fa5bd service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:22.631863 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1242/2517] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:24:22.752792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc0b59-575b-4c36-88d5-2883b6d402a8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:22.756870 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-07dc0b59-575b-4c36-88d5-2883b6d402a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:22.765733 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:22.769908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc0b59-575b-4c36-88d5-2883b6d402a8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:22.774692 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65888be0-6955-4f30-9780-446607a81f0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:22.775442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65888be0-6955-4f30-9780-446607a81f0d service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:22.775918 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65888be0-6955-4f30-9780-446607a81f0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:22.776425 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65888be0-6955-4f30-9780-446607a81f0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:22.860588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65888be0-6955-4f30-9780-446607a81f0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:22.862705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65888be0-6955-4f30-9780-446607a81f0d service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:22.874218 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1276/2518] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:23.339282 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d28645-988e-4a4a-9ba7-6dee618fa5bd service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:23.340763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a8d28645-988e-4a4a-9ba7-6dee618fa5bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:23.367774 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:23.405390 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d28645-988e-4a4a-9ba7-6dee618fa5bd service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:23.409834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d46f79-ade1-4b2a-bda3-d31830097b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:23.410447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d46f79-ade1-4b2a-bda3-d31830097b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:23.410732 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d46f79-ade1-4b2a-bda3-d31830097b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:23.411608 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d46f79-ade1-4b2a-bda3-d31830097b3d admin admin] PATH_INFO: `/v3/users/36b26040882a4e85b82b40a3c17d33bb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:23.412714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d46f79-ade1-4b2a-bda3-d31830097b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=36b26040882a4e85b82b40a3c17d33bb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:23.414621 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d46f79-ade1-4b2a-bda3-d31830097b3d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:23.590353 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-27d46f79-ade1-4b2a-bda3-d31830097b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36b26040882a4e85b82b40a3c17d33bb'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:23.591036 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-27d46f79-ade1-4b2a-bda3-d31830097b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36b26040882a4e85b82b40a3c17d33bb'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:23.603641 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-27d46f79-ade1-4b2a-bda3-d31830097b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36b26040882a4e85b82b40a3c17d33bb'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:23.657446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-27d46f79-ade1-4b2a-bda3-d31830097b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36b26040882a4e85b82b40a3c17d33bb'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:23.658419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-27d46f79-ade1-4b2a-bda3-d31830097b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36b26040882a4e85b82b40a3c17d33bb'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:23.671746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1243/2519] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:23 2022] DELETE /identity/v3/users/36b26040882a4e85b82b40a3c17d33bb => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:23.925398 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65888be0-6955-4f30-9780-446607a81f0d service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:23.937068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65888be0-6955-4f30-9780-446607a81f0d service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:23.996693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e969c0-8571-4633-846f-03a45d3a5e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:23.997329 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e969c0-8571-4633-846f-03a45d3a5e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:23.997606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e969c0-8571-4633-846f-03a45d3a5e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:23.997877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e969c0-8571-4633-846f-03a45d3a5e3a admin admin] PATH_INFO: `/v3/projects/95f59687b3b24c7f9095bcb5fb909a67` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:23.999018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e969c0-8571-4633-846f-03a45d3a5e3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=95f59687b3b24c7f9095bcb5fb909a67)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:24.000929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e969c0-8571-4633-846f-03a45d3a5e3a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:24.131434 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90e969c0-8571-4633-846f-03a45d3a5e3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95f59687b3b24c7f9095bcb5fb909a67', 'request_id': 'req-90e969c0-8571-4633-846f-03a45d3a5e3a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:24.137611 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90e969c0-8571-4633-846f-03a45d3a5e3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95f59687b3b24c7f9095bcb5fb909a67', 'request_id': 'req-90e969c0-8571-4633-846f-03a45d3a5e3a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:24.251519 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90e969c0-8571-4633-846f-03a45d3a5e3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95f59687b3b24c7f9095bcb5fb909a67', 'request_id': 'req-90e969c0-8571-4633-846f-03a45d3a5e3a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:24.263513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90e969c0-8571-4633-846f-03a45d3a5e3a admin admin] The token cache is being invalidate because project 95f59687b3b24c7f9095bcb5fb909a67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:24.263799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90e969c0-8571-4633-846f-03a45d3a5e3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:24.264678 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-90e969c0-8571-4633-846f-03a45d3a5e3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:24.266184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1277/2520] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:23 2022] DELETE /identity/v3/projects/95f59687b3b24c7f9095bcb5fb909a67 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:24.271255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae5e3e-78ed-4e85-9ded-641ef6c504a1 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:24.271557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae5e3e-78ed-4e85-9ded-641ef6c504a1 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:24.271828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae5e3e-78ed-4e85-9ded-641ef6c504a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:24.292087 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-2cae5e3e-78ed-4e85-9ded-641ef6c504a1 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:24.293284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-2cae5e3e-78ed-4e85-9ded-641ef6c504a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:24.353342 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1244/2521] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:24.361551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d11e8-43b0-41ef-b5ec-fa6c7fa0b78b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:24.361832 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d11e8-43b0-41ef-b5ec-fa6c7fa0b78b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:24.362106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d11e8-43b0-41ef-b5ec-fa6c7fa0b78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:24.384791 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e94d11e8-43b0-41ef-b5ec-fa6c7fa0b78b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:24.386013 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e94d11e8-43b0-41ef-b5ec-fa6c7fa0b78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:24.394321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1278/2522] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:24.398699 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae5e3e-78ed-4e85-9ded-641ef6c504a1 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:24.408871 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae5e3e-78ed-4e85-9ded-641ef6c504a1 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:24.413793 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6dafaa-7a81-4d0f-9d34-0e48598aae58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:24.414385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dafaa-7a81-4d0f-9d34-0e48598aae58 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:24.414656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dafaa-7a81-4d0f-9d34-0e48598aae58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:24.414921 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dafaa-7a81-4d0f-9d34-0e48598aae58 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:24.415588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6dafaa-7a81-4d0f-9d34-0e48598aae58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:24:24.415883 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6dafaa-7a81-4d0f-9d34-0e48598aae58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:24.417474 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6dafaa-7a81-4d0f-9d34-0e48598aae58 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:24.429568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1245/2523] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:24:24.433983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d11e8-43b0-41ef-b5ec-fa6c7fa0b78b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:24.446282 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d11e8-43b0-41ef-b5ec-fa6c7fa0b78b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:24.451057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6823b-e28b-4642-bb3f-5818a3e56596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:24.451657 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6823b-e28b-4642-bb3f-5818a3e56596 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:24.451930 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6823b-e28b-4642-bb3f-5818a3e56596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:24.452224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6823b-e28b-4642-bb3f-5818a3e56596 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:24.452853 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6823b-e28b-4642-bb3f-5818a3e56596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:24.454762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6823b-e28b-4642-bb3f-5818a3e56596 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:24.572393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1279/2524] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:24 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:24.576708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6dafaa-7a81-4d0f-9d34-0e48598aae58 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:24.586729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6dafaa-7a81-4d0f-9d34-0e48598aae58 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:24.613656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:24.643763 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28f9cc-acff-4e09-a56c-d5d8c7da9528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:24.644362 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28f9cc-acff-4e09-a56c-d5d8c7da9528 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:24.644644 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28f9cc-acff-4e09-a56c-d5d8c7da9528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:24.644913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28f9cc-acff-4e09-a56c-d5d8c7da9528 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:24.645546 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a28f9cc-acff-4e09-a56c-d5d8c7da9528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:24.647586 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a28f9cc-acff-4e09-a56c-d5d8c7da9528 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:24.897599 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1246/2525] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:24 2022] POST /identity/v3/users => generated 288 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:24.909705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6823b-e28b-4642-bb3f-5818a3e56596 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:24.917314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:24.921160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6823b-e28b-4642-bb3f-5818a3e56596 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:24.925819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff68ba-18de-4cf8-9806-b13356e81add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:24.926424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff68ba-18de-4cf8-9806-b13356e81add admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:24.926674 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff68ba-18de-4cf8-9806-b13356e81add admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:24.927017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff68ba-18de-4cf8-9806-b13356e81add admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:24.927813 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff68ba-18de-4cf8-9806-b13356e81add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:24.929507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff68ba-18de-4cf8-9806-b13356e81add admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:24.939543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1280/2526] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:24.943610 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28f9cc-acff-4e09-a56c-d5d8c7da9528 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:24.953678 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28f9cc-acff-4e09-a56c-d5d8c7da9528 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:24.958868 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0048b2d-7f66-445d-98ec-abe1acf84e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:24.962500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0048b2d-7f66-445d-98ec-abe1acf84e0e admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:24.962500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0048b2d-7f66-445d-98ec-abe1acf84e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:24.962500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0048b2d-7f66-445d-98ec-abe1acf84e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:24.962500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0048b2d-7f66-445d-98ec-abe1acf84e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:24.962500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0048b2d-7f66-445d-98ec-abe1acf84e0e admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:24.972668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1247/2527] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:24.977435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff68ba-18de-4cf8-9806-b13356e81add admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:24.991286 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff68ba-18de-4cf8-9806-b13356e81add admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:24.994579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb62ca-075f-4411-ac3c-cf4e1f1b0554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:24.995275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb62ca-075f-4411-ac3c-cf4e1f1b0554 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:24.995556 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb62ca-075f-4411-ac3c-cf4e1f1b0554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:24.995945 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb62ca-075f-4411-ac3c-cf4e1f1b0554 admin admin] PATH_INFO: `/v3/projects/3aad9591668d4fff82de33af929f4927/users/fbe5787703ba486f9f0ac7a299484c4e/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:25.010987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb62ca-075f-4411-ac3c-cf4e1f1b0554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aad9591668d4fff82de33af929f4927, user_id=fbe5787703ba486f9f0ac7a299484c4e, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:25.013674 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb62ca-075f-4411-ac3c-cf4e1f1b0554 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:25.124683 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1281/2528] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:24 2022] PUT /identity/v3/projects/3aad9591668d4fff82de33af929f4927/users/fbe5787703ba486f9f0ac7a299484c4e/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:25.136226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66783d7-0135-4c87-9712-3c01d8935a48 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:25.136510 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66783d7-0135-4c87-9712-3c01d8935a48 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:25.136808 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66783d7-0135-4c87-9712-3c01d8935a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:25.167100 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-b66783d7-0135-4c87-9712-3c01d8935a48 None None] MFA Rules not processed for user `fbe5787703ba486f9f0ac7a299484c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:25.168324 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b66783d7-0135-4c87-9712-3c01d8935a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:25.234428 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1248/2529] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:24:25.245914 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb62ca-075f-4411-ac3c-cf4e1f1b0554 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:25.247649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5afb62ca-075f-4411-ac3c-cf4e1f1b0554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:25.341040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb62ca-075f-4411-ac3c-cf4e1f1b0554 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:25.341040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b491482-1d3e-4402-ab71-673eca3bd725 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:25.341040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b491482-1d3e-4402-ab71-673eca3bd725 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:25.341040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b491482-1d3e-4402-ab71-673eca3bd725 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:25.341454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b491482-1d3e-4402-ab71-673eca3bd725 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:25.341454 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b491482-1d3e-4402-ab71-673eca3bd725 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:25.341638 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b491482-1d3e-4402-ab71-673eca3bd725 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:25.351256 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1282/2530] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:25.577739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66783d7-0135-4c87-9712-3c01d8935a48 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:25.643536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66783d7-0135-4c87-9712-3c01d8935a48 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:25.648115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef1258-04bf-4a81-b6f9-34e49d26b26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:25.648742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef1258-04bf-4a81-b6f9-34e49d26b26e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:25.649020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef1258-04bf-4a81-b6f9-34e49d26b26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:25.649292 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef1258-04bf-4a81-b6f9-34e49d26b26e admin admin] PATH_INFO: `/v3/users/fbe5787703ba486f9f0ac7a299484c4e` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:25.650434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef1258-04bf-4a81-b6f9-34e49d26b26e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbe5787703ba486f9f0ac7a299484c4e)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:25.652429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef1258-04bf-4a81-b6f9-34e49d26b26e admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:25.876748 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:25.877519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-deef1258-04bf-4a81-b6f9-34e49d26b26e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbe5787703ba486f9f0ac7a299484c4e'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:25.878233 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-deef1258-04bf-4a81-b6f9-34e49d26b26e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbe5787703ba486f9f0ac7a299484c4e'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:25.892069 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-deef1258-04bf-4a81-b6f9-34e49d26b26e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbe5787703ba486f9f0ac7a299484c4e'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:25.977772 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-deef1258-04bf-4a81-b6f9-34e49d26b26e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbe5787703ba486f9f0ac7a299484c4e'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:25.978454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-deef1258-04bf-4a81-b6f9-34e49d26b26e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbe5787703ba486f9f0ac7a299484c4e'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:25.992347 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1249/2531] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:25 2022] DELETE /identity/v3/users/fbe5787703ba486f9f0ac7a299484c4e => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:25.996896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1120552-9294-4e8f-b365-b8d1352c037b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:25.997381 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1120552-9294-4e8f-b365-b8d1352c037b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:25.997489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1120552-9294-4e8f-b365-b8d1352c037b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:26.017034 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b1120552-9294-4e8f-b365-b8d1352c037b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:26.018238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b1120552-9294-4e8f-b365-b8d1352c037b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:26.076901 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1283/2532] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:26.086722 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef1258-04bf-4a81-b6f9-34e49d26b26e admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:26.088169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-deef1258-04bf-4a81-b6f9-34e49d26b26e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:26.153116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef1258-04bf-4a81-b6f9-34e49d26b26e admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:26.156687 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4799e4c-148e-4843-84be-c959898b426a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:26.157291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4799e4c-148e-4843-84be-c959898b426a service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:26.157569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4799e4c-148e-4843-84be-c959898b426a service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:26.157838 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4799e4c-148e-4843-84be-c959898b426a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:26.168607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4799e4c-148e-4843-84be-c959898b426a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:26.170045 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4799e4c-148e-4843-84be-c959898b426a service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:26.180121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1250/2533] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:26.600283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1120552-9294-4e8f-b365-b8d1352c037b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:26.605406 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b1120552-9294-4e8f-b365-b8d1352c037b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:26.613707 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1120552-9294-4e8f-b365-b8d1352c037b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:26.618964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561870de-514b-4931-98b0-725a7a573ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:26.619603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561870de-514b-4931-98b0-725a7a573ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:26.619876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561870de-514b-4931-98b0-725a7a573ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:26.620140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561870de-514b-4931-98b0-725a7a573ff3 admin admin] PATH_INFO: `/v3/projects/3aad9591668d4fff82de33af929f4927` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:26.621331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561870de-514b-4931-98b0-725a7a573ff3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aad9591668d4fff82de33af929f4927)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:26.623368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561870de-514b-4931-98b0-725a7a573ff3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:26.730312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-561870de-514b-4931-98b0-725a7a573ff3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aad9591668d4fff82de33af929f4927', 'request_id': 'req-561870de-514b-4931-98b0-725a7a573ff3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:26.731405 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:26.737129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-561870de-514b-4931-98b0-725a7a573ff3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aad9591668d4fff82de33af929f4927', 'request_id': 'req-561870de-514b-4931-98b0-725a7a573ff3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:26.820578 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-561870de-514b-4931-98b0-725a7a573ff3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aad9591668d4fff82de33af929f4927', 'request_id': 'req-561870de-514b-4931-98b0-725a7a573ff3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:26.831054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-561870de-514b-4931-98b0-725a7a573ff3 admin admin] The token cache is being invalidate because project 3aad9591668d4fff82de33af929f4927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:26.831337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-561870de-514b-4931-98b0-725a7a573ff3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:26.831948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-561870de-514b-4931-98b0-725a7a573ff3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:26.834219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1284/2534] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:26 2022] DELETE /identity/v3/projects/3aad9591668d4fff82de33af929f4927 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:26.839155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ba123-5a50-4e9b-be51-ab2004de6f8f None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:26.839637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ba123-5a50-4e9b-be51-ab2004de6f8f None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:26.839732 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ba123-5a50-4e9b-be51-ab2004de6f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:26.859548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-fb8ba123-5a50-4e9b-be51-ab2004de6f8f None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:26.860740 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-fb8ba123-5a50-4e9b-be51-ab2004de6f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:26.918905 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1251/2535] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:26.926516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed373d-d7d3-4b4c-9c7b-400758c6cd89 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:26.926855 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed373d-d7d3-4b4c-9c7b-400758c6cd89 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:26.927164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed373d-d7d3-4b4c-9c7b-400758c6cd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:26.946841 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-75ed373d-d7d3-4b4c-9c7b-400758c6cd89 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:26.948128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-75ed373d-d7d3-4b4c-9c7b-400758c6cd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:26.955864 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1285/2536] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:24:26.961045 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ba123-5a50-4e9b-be51-ab2004de6f8f None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:26.977729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ba123-5a50-4e9b-be51-ab2004de6f8f None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:26.982641 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa5cc6-2924-404e-b8dc-69aecd70882a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:26.983215 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa5cc6-2924-404e-b8dc-69aecd70882a admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:26.983492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa5cc6-2924-404e-b8dc-69aecd70882a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:26.983761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa5cc6-2924-404e-b8dc-69aecd70882a admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:26.984404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa5cc6-2924-404e-b8dc-69aecd70882a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:24:26.984697 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa5cc6-2924-404e-b8dc-69aecd70882a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:26.986297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa5cc6-2924-404e-b8dc-69aecd70882a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:26.998048 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1252/2537] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:24:27.002451 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed373d-d7d3-4b4c-9c7b-400758c6cd89 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:27.012980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed373d-d7d3-4b4c-9c7b-400758c6cd89 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:27.017868 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f164b2-86f3-49b5-a56d-741139de006e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:27.018466 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f164b2-86f3-49b5-a56d-741139de006e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:27.018744 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f164b2-86f3-49b5-a56d-741139de006e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:27.019687 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f164b2-86f3-49b5-a56d-741139de006e admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:27.020394 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f164b2-86f3-49b5-a56d-741139de006e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:27.023607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f164b2-86f3-49b5-a56d-741139de006e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:27.126996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1286/2538] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:27 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:27.132035 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa5cc6-2924-404e-b8dc-69aecd70882a admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:27.142434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa5cc6-2924-404e-b8dc-69aecd70882a admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:27.187367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f164b2-86f3-49b5-a56d-741139de006e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:27.189185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c0f164b2-86f3-49b5-a56d-741139de006e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:27.192311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:27.206631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1a085-2640-45ac-95b5-7fba754b5d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:27.207265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1a085-2640-45ac-95b5-7fba754b5d9c admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:27.207543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1a085-2640-45ac-95b5-7fba754b5d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:27.207814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1a085-2640-45ac-95b5-7fba754b5d9c admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:27.208501 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1a085-2640-45ac-95b5-7fba754b5d9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:27.210576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1a085-2640-45ac-95b5-7fba754b5d9c admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:27.256359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f164b2-86f3-49b5-a56d-741139de006e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:27.261049 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb69cf-ae11-4659-ae69-4737e9e2d847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:27.261665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb69cf-ae11-4659-ae69-4737e9e2d847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:27.263527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb69cf-ae11-4659-ae69-4737e9e2d847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:27.263527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb69cf-ae11-4659-ae69-4737e9e2d847 admin admin] PATH_INFO: `/v3/users/ce62a1e880f047bf935bf31bf11c71ff` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:27.263527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb69cf-ae11-4659-ae69-4737e9e2d847 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce62a1e880f047bf935bf31bf11c71ff)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:27.265582 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb69cf-ae11-4659-ae69-4737e9e2d847 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:27.285683 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1253/2539] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:27 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:27.290603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1a085-2640-45ac-95b5-7fba754b5d9c admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:27.300799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1a085-2640-45ac-95b5-7fba754b5d9c admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:27.306785 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986a188-e374-44a2-9e9b-2262c00058e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:27.307415 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986a188-e374-44a2-9e9b-2262c00058e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:27.307744 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986a188-e374-44a2-9e9b-2262c00058e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:27.308108 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986a188-e374-44a2-9e9b-2262c00058e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:27.308759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986a188-e374-44a2-9e9b-2262c00058e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:27.310431 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986a188-e374-44a2-9e9b-2262c00058e2 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:27.322232 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1254/2540] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:24:27.326248 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986a188-e374-44a2-9e9b-2262c00058e2 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:27.336322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986a188-e374-44a2-9e9b-2262c00058e2 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:27.340925 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf301ca-f4b4-4e4a-9cbc-983f55510120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:27.341540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf301ca-f4b4-4e4a-9cbc-983f55510120 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:27.341813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf301ca-f4b4-4e4a-9cbc-983f55510120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:27.342078 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf301ca-f4b4-4e4a-9cbc-983f55510120 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:27.342742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf301ca-f4b4-4e4a-9cbc-983f55510120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:27.344396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf301ca-f4b4-4e4a-9cbc-983f55510120 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:27.354404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1255/2541] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:24:27.358271 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf301ca-f4b4-4e4a-9cbc-983f55510120 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:27.369086 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf301ca-f4b4-4e4a-9cbc-983f55510120 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:27.373825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d4ab4-6dbd-4a0f-aef9-deced56dc1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:27.374453 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d4ab4-6dbd-4a0f-aef9-deced56dc1f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:27.374711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d4ab4-6dbd-4a0f-aef9-deced56dc1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:27.375039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d4ab4-6dbd-4a0f-aef9-deced56dc1f1 admin admin] PATH_INFO: `/v3/projects/c413414f0c6f4782a2e2dc609d3c3170/users/2c27d79cf4b84f048a87a8292829ef7f/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:27.390274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d4ab4-6dbd-4a0f-aef9-deced56dc1f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c413414f0c6f4782a2e2dc609d3c3170, user_id=2c27d79cf4b84f048a87a8292829ef7f, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:27.392964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d4ab4-6dbd-4a0f-aef9-deced56dc1f1 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:27.444960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bdcb69cf-ae11-4659-ae69-4737e9e2d847 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce62a1e880f047bf935bf31bf11c71ff'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:27.456885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bdcb69cf-ae11-4659-ae69-4737e9e2d847 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce62a1e880f047bf935bf31bf11c71ff'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:27.457564 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bdcb69cf-ae11-4659-ae69-4737e9e2d847 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce62a1e880f047bf935bf31bf11c71ff'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:27.468673 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1256/2542] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:27 2022] PUT /identity/v3/projects/c413414f0c6f4782a2e2dc609d3c3170/users/2c27d79cf4b84f048a87a8292829ef7f/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:24:27.480579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09703caf-7fa6-47d2-8c66-5f0728ead032 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:27.480860 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09703caf-7fa6-47d2-8c66-5f0728ead032 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:27.481148 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09703caf-7fa6-47d2-8c66-5f0728ead032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:27.509039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-09703caf-7fa6-47d2-8c66-5f0728ead032 None None] MFA Rules not processed for user `2c27d79cf4b84f048a87a8292829ef7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:27.510177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-09703caf-7fa6-47d2-8c66-5f0728ead032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:27.531572 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bdcb69cf-ae11-4659-ae69-4737e9e2d847 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce62a1e880f047bf935bf31bf11c71ff'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:27.532313 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bdcb69cf-ae11-4659-ae69-4737e9e2d847 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce62a1e880f047bf935bf31bf11c71ff'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:27.547698 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1287/2543] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:27 2022] DELETE /identity/v3/users/ce62a1e880f047bf935bf31bf11c71ff => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:27.554296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0b894-3238-4db4-9ed8-a94ffbde6025 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:27.554659 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0b894-3238-4db4-9ed8-a94ffbde6025 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:27.554932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0b894-3238-4db4-9ed8-a94ffbde6025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:27.574223 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a8c0b894-3238-4db4-9ed8-a94ffbde6025 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:27.575449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a8c0b894-3238-4db4-9ed8-a94ffbde6025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:27.579872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1257/2544] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:24:27.584712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09703caf-7fa6-47d2-8c66-5f0728ead032 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:27.586513 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-09703caf-7fa6-47d2-8c66-5f0728ead032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:27.600382 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=84,keystone:INSERT=3,keystone:DELETE=13 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:27.636396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1288/2545] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:27.647089 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0b894-3238-4db4-9ed8-a94ffbde6025 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:27.648393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a8c0b894-3238-4db4-9ed8-a94ffbde6025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:27.654169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09703caf-7fa6-47d2-8c66-5f0728ead032 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:27.658887 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f3477-a7b1-49b0-906f-266164b45a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:27.659515 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f3477-a7b1-49b0-906f-266164b45a94 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:27.659794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f3477-a7b1-49b0-906f-266164b45a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:27.660061 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f3477-a7b1-49b0-906f-266164b45a94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:27.660716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f3477-a7b1-49b0-906f-266164b45a94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:27.663345 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f3477-a7b1-49b0-906f-266164b45a94 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:27.706710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0b894-3238-4db4-9ed8-a94ffbde6025 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:27.710961 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8ef24-774f-45d5-a1e5-aaa71ea0368e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:27.711603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8ef24-774f-45d5-a1e5-aaa71ea0368e service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:27.712228 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8ef24-774f-45d5-a1e5-aaa71ea0368e service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:27.712409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8ef24-774f-45d5-a1e5-aaa71ea0368e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:27.721290 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8ef24-774f-45d5-a1e5-aaa71ea0368e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:27.722713 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8ef24-774f-45d5-a1e5-aaa71ea0368e service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:27.733679 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1289/2546] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:24:27.771376 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1258/2547] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:27 2022] POST /identity/v3/projects => generated 365 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:24:27.772261 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8ef24-774f-45d5-a1e5-aaa71ea0368e service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:27.782018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8ef24-774f-45d5-a1e5-aaa71ea0368e service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:27.840717 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87e33f-23a1-46bc-b5d8-c26f37b17ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:27.841315 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87e33f-23a1-46bc-b5d8-c26f37b17ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:27.841588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87e33f-23a1-46bc-b5d8-c26f37b17ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:27.841851 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87e33f-23a1-46bc-b5d8-c26f37b17ab1 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:27.842467 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87e33f-23a1-46bc-b5d8-c26f37b17ab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:27.844452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87e33f-23a1-46bc-b5d8-c26f37b17ab1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:27.968879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1290/2548] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:27 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:27.972116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f3477-a7b1-49b0-906f-266164b45a94 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:27.979986 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:27.984053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f3477-a7b1-49b0-906f-266164b45a94 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:27.989348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156aa331-701f-4aa8-a325-2ca9aa36eaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:27.990049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aa331-701f-4aa8-a325-2ca9aa36eaa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:27.990396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aa331-701f-4aa8-a325-2ca9aa36eaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:27.990735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aa331-701f-4aa8-a325-2ca9aa36eaa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:27.991790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156aa331-701f-4aa8-a325-2ca9aa36eaa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:27.993168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156aa331-701f-4aa8-a325-2ca9aa36eaa6 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:28.003303 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1259/2549] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:28.007748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87e33f-23a1-46bc-b5d8-c26f37b17ab1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:28.018165 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87e33f-23a1-46bc-b5d8-c26f37b17ab1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:28.023847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61970d8-15e5-4164-ba0f-774e0b834140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:28.024582 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61970d8-15e5-4164-ba0f-774e0b834140 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:28.024925 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61970d8-15e5-4164-ba0f-774e0b834140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:28.025389 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61970d8-15e5-4164-ba0f-774e0b834140 admin admin] PATH_INFO: `/v3/projects/9832efd6a2f54c0bad422d42575416eb/users/3f136ce3fa644465abcbace626da90de/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:28.041704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61970d8-15e5-4164-ba0f-774e0b834140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9832efd6a2f54c0bad422d42575416eb, user_id=3f136ce3fa644465abcbace626da90de, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:28.044621 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61970d8-15e5-4164-ba0f-774e0b834140 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:28.112875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1291/2550] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:28 2022] PUT /identity/v3/projects/9832efd6a2f54c0bad422d42575416eb/users/3f136ce3fa644465abcbace626da90de/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:28.117575 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156aa331-701f-4aa8-a325-2ca9aa36eaa6 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:28.128940 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156aa331-701f-4aa8-a325-2ca9aa36eaa6 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:28.176954 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61970d8-15e5-4164-ba0f-774e0b834140 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:28.178854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b61970d8-15e5-4164-ba0f-774e0b834140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:28.193043 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1addd5-8459-4ef0-8642-9f83046100f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:28.194545 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1addd5-8459-4ef0-8642-9f83046100f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:28.194545 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1addd5-8459-4ef0-8642-9f83046100f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:28.194766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1addd5-8459-4ef0-8642-9f83046100f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:28.195385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1addd5-8459-4ef0-8642-9f83046100f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:28.201046 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1addd5-8459-4ef0-8642-9f83046100f3 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:28.214903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1260/2551] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:28.218426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1addd5-8459-4ef0-8642-9f83046100f3 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:28.229468 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1addd5-8459-4ef0-8642-9f83046100f3 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:28.230145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:28.235027 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9fd56b-9dfc-4ed1-b818-c2a05f2ec56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:28.235764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fd56b-9dfc-4ed1-b818-c2a05f2ec56d admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:28.236114 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fd56b-9dfc-4ed1-b818-c2a05f2ec56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:28.236396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fd56b-9dfc-4ed1-b818-c2a05f2ec56d admin admin] PATH_INFO: `/v3/projects/9832efd6a2f54c0bad422d42575416eb/users/3f136ce3fa644465abcbace626da90de/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:28.238460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9fd56b-9dfc-4ed1-b818-c2a05f2ec56d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9832efd6a2f54c0bad422d42575416eb, user_id=3f136ce3fa644465abcbace626da90de, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:28.241270 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9fd56b-9dfc-4ed1-b818-c2a05f2ec56d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:28.251675 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1261/2552] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:28 2022] PUT /identity/v3/projects/9832efd6a2f54c0bad422d42575416eb/users/3f136ce3fa644465abcbace626da90de/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:24:28.252748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61970d8-15e5-4164-ba0f-774e0b834140 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:28.262881 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982ae04-0958-4643-a3a2-7813585d0318 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:28.263195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982ae04-0958-4643-a3a2-7813585d0318 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:28.263466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982ae04-0958-4643-a3a2-7813585d0318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:28.293962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-e982ae04-0958-4643-a3a2-7813585d0318 None None] MFA Rules not processed for user `3f136ce3fa644465abcbace626da90de`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:28.295136 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e982ae04-0958-4643-a3a2-7813585d0318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:28.310368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:28.311040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:28.311309 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:28.311576 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd admin admin] PATH_INFO: `/v3/projects/90ebfd98a43043a19dd06ebd2c96cd17` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:28.312734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=90ebfd98a43043a19dd06ebd2c96cd17)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:28.314983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:28.367342 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1262/2553] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:24:28.378759 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982ae04-0958-4643-a3a2-7813585d0318 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:28.380767 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e982ae04-0958-4643-a3a2-7813585d0318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:28.437150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982ae04-0958-4643-a3a2-7813585d0318 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:28.440892 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1961c76-00f4-4ab9-90b8-1421c14137d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:28.441486 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1961c76-00f4-4ab9-90b8-1421c14137d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:28.441766 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1961c76-00f4-4ab9-90b8-1421c14137d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:28.442037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1961c76-00f4-4ab9-90b8-1421c14137d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:28.451996 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1961c76-00f4-4ab9-90b8-1421c14137d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:28.458696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1961c76-00f4-4ab9-90b8-1421c14137d2 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:28.463833 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90ebfd98a43043a19dd06ebd2c96cd17', 'request_id': 'req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:28.464029 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1263/2554] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:28 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:24:28.470580 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90ebfd98a43043a19dd06ebd2c96cd17', 'request_id': 'req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:28.552913 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90ebfd98a43043a19dd06ebd2c96cd17', 'request_id': 'req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:28.564453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd admin admin] The token cache is being invalidate because project 90ebfd98a43043a19dd06ebd2c96cd17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:28.564739 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:28.565426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:28.566969 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1292/2555] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:28 2022] DELETE /identity/v3/projects/90ebfd98a43043a19dd06ebd2c96cd17 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:28.571995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1961c76-00f4-4ab9-90b8-1421c14137d2 service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:28.578183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:28.582517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1961c76-00f4-4ab9-90b8-1421c14137d2 service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:28.641494 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b640b-0ca6-41b6-8223-2a53870545be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:28.642489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b640b-0ca6-41b6-8223-2a53870545be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:28.642816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b640b-0ca6-41b6-8223-2a53870545be admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:28.643124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b640b-0ca6-41b6-8223-2a53870545be admin admin] PATH_INFO: `/v3/users/b7e6b462a2cc4bf5a867fbcac3131b85` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:28.644274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b640b-0ca6-41b6-8223-2a53870545be admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e6b462a2cc4bf5a867fbcac3131b85)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:28.646801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b640b-0ca6-41b6-8223-2a53870545be admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:28.717396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:28.718898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:28.778590 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f056c01-028e-4a7e-b64f-c12d1fb5d7cd admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:28.782529 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122316e1-d1d8-4dc1-92da-9bff1e281f9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:28.783193 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122316e1-d1d8-4dc1-92da-9bff1e281f9f service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:28.783637 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122316e1-d1d8-4dc1-92da-9bff1e281f9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:28.783830 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122316e1-d1d8-4dc1-92da-9bff1e281f9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:28.785626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-122316e1-d1d8-4dc1-92da-9bff1e281f9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:28.817504 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:28.845949 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122316e1-d1d8-4dc1-92da-9bff1e281f9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:28.851084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122316e1-d1d8-4dc1-92da-9bff1e281f9f service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:28.852593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-836b640b-0ca6-41b6-8223-2a53870545be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e6b462a2cc4bf5a867fbcac3131b85'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:28.853257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-836b640b-0ca6-41b6-8223-2a53870545be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e6b462a2cc4bf5a867fbcac3131b85'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:28.857915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1293/2556] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:24:28.866302 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-836b640b-0ca6-41b6-8223-2a53870545be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e6b462a2cc4bf5a867fbcac3131b85'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:28.946475 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-836b640b-0ca6-41b6-8223-2a53870545be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e6b462a2cc4bf5a867fbcac3131b85'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:28.947170 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-836b640b-0ca6-41b6-8223-2a53870545be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e6b462a2cc4bf5a867fbcac3131b85'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:28.960068 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1264/2557] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:28 2022] DELETE /identity/v3/users/b7e6b462a2cc4bf5a867fbcac3131b85 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:29.378632 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122316e1-d1d8-4dc1-92da-9bff1e281f9f service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:29.380717 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-122316e1-d1d8-4dc1-92da-9bff1e281f9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:29.394814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122316e1-d1d8-4dc1-92da-9bff1e281f9f service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:29.455200 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54737e9-349d-4012-9477-ecc5dd1d7836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:29.455874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54737e9-349d-4012-9477-ecc5dd1d7836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:29.456147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54737e9-349d-4012-9477-ecc5dd1d7836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:29.456440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54737e9-349d-4012-9477-ecc5dd1d7836 admin admin] PATH_INFO: `/v3/projects/2b4fd4d2a3b64812b23999ece1c402c7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:29.457560 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54737e9-349d-4012-9477-ecc5dd1d7836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b4fd4d2a3b64812b23999ece1c402c7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:29.460083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54737e9-349d-4012-9477-ecc5dd1d7836 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:29.588701 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f54737e9-349d-4012-9477-ecc5dd1d7836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b4fd4d2a3b64812b23999ece1c402c7', 'request_id': 'req-f54737e9-349d-4012-9477-ecc5dd1d7836'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:29.594979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f54737e9-349d-4012-9477-ecc5dd1d7836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b4fd4d2a3b64812b23999ece1c402c7', 'request_id': 'req-f54737e9-349d-4012-9477-ecc5dd1d7836'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:29.645528 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f54737e9-349d-4012-9477-ecc5dd1d7836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b4fd4d2a3b64812b23999ece1c402c7', 'request_id': 'req-f54737e9-349d-4012-9477-ecc5dd1d7836'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:29.656677 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f54737e9-349d-4012-9477-ecc5dd1d7836 admin admin] The token cache is being invalidate because project 2b4fd4d2a3b64812b23999ece1c402c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:29.656958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f54737e9-349d-4012-9477-ecc5dd1d7836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:29.657588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f54737e9-349d-4012-9477-ecc5dd1d7836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:29.659137 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1294/2558] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:29 2022] DELETE /identity/v3/projects/2b4fd4d2a3b64812b23999ece1c402c7 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:29.663338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b640b-0ca6-41b6-8223-2a53870545be admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:29.664816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-836b640b-0ca6-41b6-8223-2a53870545be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:29.678024 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b640b-0ca6-41b6-8223-2a53870545be admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:29.735378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b7fce-c7c0-43f1-9904-3e558205a20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:29.736011 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b7fce-c7c0-43f1-9904-3e558205a20a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:29.736278 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b7fce-c7c0-43f1-9904-3e558205a20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:29.736553 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b7fce-c7c0-43f1-9904-3e558205a20a admin admin] PATH_INFO: `/v3/users/006b5eab31ef455ba8de0e1b311151c7` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:29.737696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b7fce-c7c0-43f1-9904-3e558205a20a admin admin] RBAC: Authorizing `identity:delete_user(user_id=006b5eab31ef455ba8de0e1b311151c7)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:29.740728 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b7fce-c7c0-43f1-9904-3e558205a20a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:29.861734 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:29.967614 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-cb2b7fce-c7c0-43f1-9904-3e558205a20a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '006b5eab31ef455ba8de0e1b311151c7'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:29.968202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-cb2b7fce-c7c0-43f1-9904-3e558205a20a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '006b5eab31ef455ba8de0e1b311151c7'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:29.980454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-cb2b7fce-c7c0-43f1-9904-3e558205a20a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '006b5eab31ef455ba8de0e1b311151c7'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:30.050712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-cb2b7fce-c7c0-43f1-9904-3e558205a20a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '006b5eab31ef455ba8de0e1b311151c7'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:30.051358 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-cb2b7fce-c7c0-43f1-9904-3e558205a20a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '006b5eab31ef455ba8de0e1b311151c7'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:30.064545 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1265/2559] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:29 2022] DELETE /identity/v3/users/006b5eab31ef455ba8de0e1b311151c7 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:30.527815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54737e9-349d-4012-9477-ecc5dd1d7836 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:30.529842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f54737e9-349d-4012-9477-ecc5dd1d7836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:30.542761 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54737e9-349d-4012-9477-ecc5dd1d7836 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:30.601458 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b111683-0593-4ffe-8278-1b98db5ee1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:30.602080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b111683-0593-4ffe-8278-1b98db5ee1b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:30.602352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b111683-0593-4ffe-8278-1b98db5ee1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:30.602645 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b111683-0593-4ffe-8278-1b98db5ee1b3 admin admin] PATH_INFO: `/v3/projects/81b276a06a44467695cd7d498ea27bc2` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:30.603784 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b111683-0593-4ffe-8278-1b98db5ee1b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81b276a06a44467695cd7d498ea27bc2)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:30.605757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b111683-0593-4ffe-8278-1b98db5ee1b3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:30.607207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:30.760639 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6b111683-0593-4ffe-8278-1b98db5ee1b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81b276a06a44467695cd7d498ea27bc2', 'request_id': 'req-6b111683-0593-4ffe-8278-1b98db5ee1b3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:30.766931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6b111683-0593-4ffe-8278-1b98db5ee1b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81b276a06a44467695cd7d498ea27bc2', 'request_id': 'req-6b111683-0593-4ffe-8278-1b98db5ee1b3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:30.839884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6b111683-0593-4ffe-8278-1b98db5ee1b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81b276a06a44467695cd7d498ea27bc2', 'request_id': 'req-6b111683-0593-4ffe-8278-1b98db5ee1b3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:30.850643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6b111683-0593-4ffe-8278-1b98db5ee1b3 admin admin] The token cache is being invalidate because project 81b276a06a44467695cd7d498ea27bc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:30.850924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6b111683-0593-4ffe-8278-1b98db5ee1b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:30.851561 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6b111683-0593-4ffe-8278-1b98db5ee1b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:30.853053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1295/2560] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:30 2022] DELETE /identity/v3/projects/81b276a06a44467695cd7d498ea27bc2 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:30.859229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cf840-832d-4e8b-b44e-25bc8311652e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:30.859699 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cf840-832d-4e8b-b44e-25bc8311652e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:30.859816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cf840-832d-4e8b-b44e-25bc8311652e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:30.887138 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-e75cf840-832d-4e8b-b44e-25bc8311652e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:30.888343 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e75cf840-832d-4e8b-b44e-25bc8311652e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:30.948707 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1266/2561] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:30.958772 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11f8b7-0166-47cd-80c6-a94d2c6874ee None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:30.959097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11f8b7-0166-47cd-80c6-a94d2c6874ee None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:30.959374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11f8b7-0166-47cd-80c6-a94d2c6874ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:30.980269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5d11f8b7-0166-47cd-80c6-a94d2c6874ee None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:30.982211 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5d11f8b7-0166-47cd-80c6-a94d2c6874ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:30.996235 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1296/2562] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:31.000658 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cf840-832d-4e8b-b44e-25bc8311652e None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:31.011706 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cf840-832d-4e8b-b44e-25bc8311652e None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:31.017603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f398260-87d6-4f02-9f98-25a4e447d5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:31.018209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f398260-87d6-4f02-9f98-25a4e447d5ac admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:31.018544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f398260-87d6-4f02-9f98-25a4e447d5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:31.019550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f398260-87d6-4f02-9f98-25a4e447d5ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:31.020242 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f398260-87d6-4f02-9f98-25a4e447d5ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:24:31.020555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f398260-87d6-4f02-9f98-25a4e447d5ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:31.022232 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f398260-87d6-4f02-9f98-25a4e447d5ac admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:31.035742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1267/2563] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:24:31.040795 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11f8b7-0166-47cd-80c6-a94d2c6874ee None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:31.050216 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11f8b7-0166-47cd-80c6-a94d2c6874ee None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:31.055525 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a669e79-1a7f-4dba-8172-2af7d6f48f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:31.056116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a669e79-1a7f-4dba-8172-2af7d6f48f7d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:31.056387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a669e79-1a7f-4dba-8172-2af7d6f48f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:31.056702 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a669e79-1a7f-4dba-8172-2af7d6f48f7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:31.057332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a669e79-1a7f-4dba-8172-2af7d6f48f7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:31.059292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a669e79-1a7f-4dba-8172-2af7d6f48f7d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:31.155416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1297/2564] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:31 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:31.159413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f398260-87d6-4f02-9f98-25a4e447d5ac admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:31.170236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f398260-87d6-4f02-9f98-25a4e447d5ac admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:31.226589 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d316fd-31ef-491a-bb37-fcde62195efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:31.227254 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d316fd-31ef-491a-bb37-fcde62195efe admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:31.227532 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d316fd-31ef-491a-bb37-fcde62195efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:31.227802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d316fd-31ef-491a-bb37-fcde62195efe admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:31.228437 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d316fd-31ef-491a-bb37-fcde62195efe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:31.230377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d316fd-31ef-491a-bb37-fcde62195efe admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:31.321661 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1268/2565] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:31 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:31.324921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a669e79-1a7f-4dba-8172-2af7d6f48f7d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:31.335377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a669e79-1a7f-4dba-8172-2af7d6f48f7d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:31.339971 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481c816-1942-4514-836d-1e2787eab874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:31.340557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481c816-1942-4514-836d-1e2787eab874 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:31.340833 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481c816-1942-4514-836d-1e2787eab874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:31.341118 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481c816-1942-4514-836d-1e2787eab874 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:31.341912 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481c816-1942-4514-836d-1e2787eab874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:31.344575 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481c816-1942-4514-836d-1e2787eab874 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:31.355186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1298/2566] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:31.360183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d316fd-31ef-491a-bb37-fcde62195efe admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:31.370844 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d316fd-31ef-491a-bb37-fcde62195efe admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:31.375539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db1373-a074-48b8-a305-f92576fc71b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:31.376123 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db1373-a074-48b8-a305-f92576fc71b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:31.376395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db1373-a074-48b8-a305-f92576fc71b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:31.376660 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db1373-a074-48b8-a305-f92576fc71b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:31.377284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db1373-a074-48b8-a305-f92576fc71b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:31.378879 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db1373-a074-48b8-a305-f92576fc71b0 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:31.391051 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1269/2567] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:31.395527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481c816-1942-4514-836d-1e2787eab874 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:31.405788 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481c816-1942-4514-836d-1e2787eab874 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:31.410620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddb2a6-8acd-4a89-befc-22fe16a0df8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:31.411825 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddb2a6-8acd-4a89-befc-22fe16a0df8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:31.412098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddb2a6-8acd-4a89-befc-22fe16a0df8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:31.412364 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddb2a6-8acd-4a89-befc-22fe16a0df8b admin admin] PATH_INFO: `/v3/projects/f171e79b21624734a13652fb85475ab7/users/b3835d1937e1459a8981bb5a5bc16281/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:31.428167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddb2a6-8acd-4a89-befc-22fe16a0df8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f171e79b21624734a13652fb85475ab7, user_id=b3835d1937e1459a8981bb5a5bc16281, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:31.430859 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddb2a6-8acd-4a89-befc-22fe16a0df8b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:31.497327 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1299/2568] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:31 2022] PUT /identity/v3/projects/f171e79b21624734a13652fb85475ab7/users/b3835d1937e1459a8981bb5a5bc16281/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:31.502446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdc364-dbd5-4287-944c-c8ec01ffaafc None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:31.502862 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdc364-dbd5-4287-944c-c8ec01ffaafc None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:31.503240 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdc364-dbd5-4287-944c-c8ec01ffaafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:31.522762 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-aebdc364-dbd5-4287-944c-c8ec01ffaafc None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:31.524056 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-aebdc364-dbd5-4287-944c-c8ec01ffaafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:31.527965 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:31.583657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1270/2569] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:31.593559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddb2a6-8acd-4a89-befc-22fe16a0df8b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:31.595332 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-80ddb2a6-8acd-4a89-befc-22fe16a0df8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:31.653849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddb2a6-8acd-4a89-befc-22fe16a0df8b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:31.657458 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58880ab8-3b4e-4a8a-8fa8-d3cbea28c285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:31.658048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58880ab8-3b4e-4a8a-8fa8-d3cbea28c285 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:31.658349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58880ab8-3b4e-4a8a-8fa8-d3cbea28c285 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:31.658611 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58880ab8-3b4e-4a8a-8fa8-d3cbea28c285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:31.668206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58880ab8-3b4e-4a8a-8fa8-d3cbea28c285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:31.669616 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58880ab8-3b4e-4a8a-8fa8-d3cbea28c285 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:31.679161 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1300/2570] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:33.040322 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366dbdb-0f0f-4384-8c93-0b2be87d66ec None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:33.040854 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366dbdb-0f0f-4384-8c93-0b2be87d66ec None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:33.040854 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366dbdb-0f0f-4384-8c93-0b2be87d66ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:33.069233 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-6366dbdb-0f0f-4384-8c93-0b2be87d66ec None None] MFA Rules not processed for user `b3835d1937e1459a8981bb5a5bc16281`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:33.070596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6366dbdb-0f0f-4384-8c93-0b2be87d66ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:33.144679 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1271/2571] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:33.149763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58880ab8-3b4e-4a8a-8fa8-d3cbea28c285 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:33.160233 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58880ab8-3b4e-4a8a-8fa8-d3cbea28c285 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:33.165628 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b445fa-2014-4c31-8a56-642decd37a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:33.166268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b445fa-2014-4c31-8a56-642decd37a53 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:33.166523 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b445fa-2014-4c31-8a56-642decd37a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:33.166789 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b445fa-2014-4c31-8a56-642decd37a53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:33.167433 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b445fa-2014-4c31-8a56-642decd37a53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:33.169314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b445fa-2014-4c31-8a56-642decd37a53 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:33.256350 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1301/2572] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:33 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:33.260816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366dbdb-0f0f-4384-8c93-0b2be87d66ec None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:33.269962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366dbdb-0f0f-4384-8c93-0b2be87d66ec None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:33.338243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056a3de-8057-437f-9841-6d867bc1304c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:33.338735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056a3de-8057-437f-9841-6d867bc1304c admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:33.339050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056a3de-8057-437f-9841-6d867bc1304c admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:33.339323 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056a3de-8057-437f-9841-6d867bc1304c admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:33.340293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056a3de-8057-437f-9841-6d867bc1304c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:33.342243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056a3de-8057-437f-9841-6d867bc1304c admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:33.447475 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1272/2573] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:33 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:33.451624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b445fa-2014-4c31-8a56-642decd37a53 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:33.462618 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b445fa-2014-4c31-8a56-642decd37a53 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:33.467124 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9fcd8-c4a9-4a5c-8167-1a2c951973b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:33.473008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9fcd8-c4a9-4a5c-8167-1a2c951973b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:33.473008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9fcd8-c4a9-4a5c-8167-1a2c951973b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:33.473008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9fcd8-c4a9-4a5c-8167-1a2c951973b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:33.473008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d9fcd8-c4a9-4a5c-8167-1a2c951973b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:33.473008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d9fcd8-c4a9-4a5c-8167-1a2c951973b0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:33.473008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:33.481396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1302/2574] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:33.485580 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056a3de-8057-437f-9841-6d867bc1304c admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:33.495598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056a3de-8057-437f-9841-6d867bc1304c admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:33.500118 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31413bbf-90b5-412b-997c-b13bf4d96e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:33.501619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31413bbf-90b5-412b-997c-b13bf4d96e4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:33.501619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31413bbf-90b5-412b-997c-b13bf4d96e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:33.501619 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31413bbf-90b5-412b-997c-b13bf4d96e4f admin admin] PATH_INFO: `/v3/projects/81d49bb855fa4f0cb2fae0cbcc2331f3/users/9f3ad007b3d2495a892ea7c236f0c3fc/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:33.516381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31413bbf-90b5-412b-997c-b13bf4d96e4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=81d49bb855fa4f0cb2fae0cbcc2331f3, user_id=9f3ad007b3d2495a892ea7c236f0c3fc, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:33.519107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31413bbf-90b5-412b-997c-b13bf4d96e4f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:33.564890 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1273/2575] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:33 2022] PUT /identity/v3/projects/81d49bb855fa4f0cb2fae0cbcc2331f3/users/9f3ad007b3d2495a892ea7c236f0c3fc/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:33.569324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9fcd8-c4a9-4a5c-8167-1a2c951973b0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:33.579096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9fcd8-c4a9-4a5c-8167-1a2c951973b0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:33.635489 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870f4d0-4e9f-40fc-952d-6cb541e6bef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:33.636118 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870f4d0-4e9f-40fc-952d-6cb541e6bef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:33.636397 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870f4d0-4e9f-40fc-952d-6cb541e6bef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:33.636671 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870f4d0-4e9f-40fc-952d-6cb541e6bef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:33.637309 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870f4d0-4e9f-40fc-952d-6cb541e6bef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:33.638919 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870f4d0-4e9f-40fc-952d-6cb541e6bef4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:33.649645 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1303/2576] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:24:33.653822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31413bbf-90b5-412b-997c-b13bf4d96e4f admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:33.663200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:33.667009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31413bbf-90b5-412b-997c-b13bf4d96e4f admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:33.671724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78849dde-4ca4-4e3b-a200-317a0662989d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:33.672385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78849dde-4ca4-4e3b-a200-317a0662989d admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:33.675067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78849dde-4ca4-4e3b-a200-317a0662989d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:33.675067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78849dde-4ca4-4e3b-a200-317a0662989d admin admin] PATH_INFO: `/v3/projects/81d49bb855fa4f0cb2fae0cbcc2331f3/users/9f3ad007b3d2495a892ea7c236f0c3fc/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:33.675067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78849dde-4ca4-4e3b-a200-317a0662989d admin admin] RBAC: Authorizing `identity:create_grant(project_id=81d49bb855fa4f0cb2fae0cbcc2331f3, user_id=9f3ad007b3d2495a892ea7c236f0c3fc, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:33.677863 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78849dde-4ca4-4e3b-a200-317a0662989d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:33.688847 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1274/2577] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:33 2022] PUT /identity/v3/projects/81d49bb855fa4f0cb2fae0cbcc2331f3/users/9f3ad007b3d2495a892ea7c236f0c3fc/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:34.925869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d40782-ab91-43f3-8568-db0804203877 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:34.926371 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d40782-ab91-43f3-8568-db0804203877 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:34.926703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d40782-ab91-43f3-8568-db0804203877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:34.957578 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-70d40782-ab91-43f3-8568-db0804203877 None None] MFA Rules not processed for user `9f3ad007b3d2495a892ea7c236f0c3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:34.959477 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-70d40782-ab91-43f3-8568-db0804203877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:35.031555 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1304/2578] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:35.044203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78849dde-4ca4-4e3b-a200-317a0662989d admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:35.045903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-78849dde-4ca4-4e3b-a200-317a0662989d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:35.104590 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78849dde-4ca4-4e3b-a200-317a0662989d admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:35.108419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56437a-17bc-4d33-98c7-812d7ada9f10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:35.109019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56437a-17bc-4d33-98c7-812d7ada9f10 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:35.109291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56437a-17bc-4d33-98c7-812d7ada9f10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:35.109557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56437a-17bc-4d33-98c7-812d7ada9f10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:35.170288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56437a-17bc-4d33-98c7-812d7ada9f10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:35.178383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56437a-17bc-4d33-98c7-812d7ada9f10 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:35.181577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1275/2579] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:36.790700 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d40782-ab91-43f3-8568-db0804203877 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:36.848415 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d40782-ab91-43f3-8568-db0804203877 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:36.853400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4333095-df91-48ed-aa2b-b797db373fb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:36.854092 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4333095-df91-48ed-aa2b-b797db373fb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:36.854502 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4333095-df91-48ed-aa2b-b797db373fb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:36.854839 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4333095-df91-48ed-aa2b-b797db373fb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:36.856820 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f4333095-df91-48ed-aa2b-b797db373fb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:36.867688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4333095-df91-48ed-aa2b-b797db373fb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:36.869207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4333095-df91-48ed-aa2b-b797db373fb2 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:36.878012 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1305/2580] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:39.129009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56437a-17bc-4d33-98c7-812d7ada9f10 service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:39.138665 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56437a-17bc-4d33-98c7-812d7ada9f10 service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:39.142666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516fb8a-5cb1-425d-9732-62db7791b41e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:39.143316 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516fb8a-5cb1-425d-9732-62db7791b41e service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:39.143587 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516fb8a-5cb1-425d-9732-62db7791b41e service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:39.143855 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516fb8a-5cb1-425d-9732-62db7791b41e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:39.168306 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516fb8a-5cb1-425d-9732-62db7791b41e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:39.169869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516fb8a-5cb1-425d-9732-62db7791b41e service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:39.183540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1276/2581] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:24:42.464876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4333095-df91-48ed-aa2b-b797db373fb2 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:42.466400 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f4333095-df91-48ed-aa2b-b797db373fb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:42.496920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:42.566889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4333095-df91-48ed-aa2b-b797db373fb2 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:42.572905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f49ac-28ff-4b7e-8afc-b334a15fa050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:42.574033 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f49ac-28ff-4b7e-8afc-b334a15fa050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:42.578860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f49ac-28ff-4b7e-8afc-b334a15fa050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:42.578860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f49ac-28ff-4b7e-8afc-b334a15fa050 admin admin] PATH_INFO: `/v3/users/2c27d79cf4b84f048a87a8292829ef7f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:42.578860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f49ac-28ff-4b7e-8afc-b334a15fa050 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c27d79cf4b84f048a87a8292829ef7f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:42.579425 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f49ac-28ff-4b7e-8afc-b334a15fa050 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:42.773299 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-962f49ac-28ff-4b7e-8afc-b334a15fa050 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c27d79cf4b84f048a87a8292829ef7f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:42.789347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-962f49ac-28ff-4b7e-8afc-b334a15fa050 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c27d79cf4b84f048a87a8292829ef7f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:42.790128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-962f49ac-28ff-4b7e-8afc-b334a15fa050 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c27d79cf4b84f048a87a8292829ef7f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:42.849839 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-962f49ac-28ff-4b7e-8afc-b334a15fa050 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c27d79cf4b84f048a87a8292829ef7f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:42.851120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-962f49ac-28ff-4b7e-8afc-b334a15fa050 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c27d79cf4b84f048a87a8292829ef7f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:42.866735 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1306/2582] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:42 2022] DELETE /identity/v3/users/2c27d79cf4b84f048a87a8292829ef7f => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:42.871412 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fb729-0830-4e43-a934-543676d20a2c None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:42.871926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fb729-0830-4e43-a934-543676d20a2c None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:42.871926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fb729-0830-4e43-a934-543676d20a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:42.896332 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-476fb729-0830-4e43-a934-543676d20a2c None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:42.898874 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-476fb729-0830-4e43-a934-543676d20a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:42.965835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1277/2583] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:42.976727 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f49ac-28ff-4b7e-8afc-b334a15fa050 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:42.978151 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-962f49ac-28ff-4b7e-8afc-b334a15fa050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:43.042947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f49ac-28ff-4b7e-8afc-b334a15fa050 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:43.048922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afc42a-d3ef-4653-92a3-2c807b24fe1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:43.048922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afc42a-d3ef-4653-92a3-2c807b24fe1f service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:43.048922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afc42a-d3ef-4653-92a3-2c807b24fe1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:43.048922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afc42a-d3ef-4653-92a3-2c807b24fe1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:43.058530 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afc42a-d3ef-4653-92a3-2c807b24fe1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:43.060041 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afc42a-d3ef-4653-92a3-2c807b24fe1f service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:43.074219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1307/2584] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:43.493172 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476fb729-0830-4e43-a934-543676d20a2c None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:43.494702 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-476fb729-0830-4e43-a934-543676d20a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:43.508787 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476fb729-0830-4e43-a934-543676d20a2c None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:43.513713 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:43.517989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:43.517989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:43.517989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d admin admin] PATH_INFO: `/v3/projects/c413414f0c6f4782a2e2dc609d3c3170` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:43.517989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c413414f0c6f4782a2e2dc609d3c3170)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:43.518454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:43.608390 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:43.623960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c413414f0c6f4782a2e2dc609d3c3170', 'request_id': 'req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:43.630155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c413414f0c6f4782a2e2dc609d3c3170', 'request_id': 'req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:43.706819 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c413414f0c6f4782a2e2dc609d3c3170', 'request_id': 'req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:43.719445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d admin admin] The token cache is being invalidate because project c413414f0c6f4782a2e2dc609d3c3170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:43.719810 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:43.720484 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:43.721974 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1278/2585] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:43 2022] DELETE /identity/v3/projects/c413414f0c6f4782a2e2dc609d3c3170 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:43.726143 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afc42a-d3ef-4653-92a3-2c807b24fe1f service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:43.736088 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afc42a-d3ef-4653-92a3-2c807b24fe1f service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:43.787361 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:43.805887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f32da-37a9-4dcb-841c-7e7df107c800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:43.808796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f32da-37a9-4dcb-841c-7e7df107c800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:43.808796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f32da-37a9-4dcb-841c-7e7df107c800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:43.808796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f32da-37a9-4dcb-841c-7e7df107c800 admin admin] PATH_INFO: `/v3/users/3f136ce3fa644465abcbace626da90de` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:43.808796 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1f32da-37a9-4dcb-841c-7e7df107c800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f136ce3fa644465abcbace626da90de)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:43.810282 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1f32da-37a9-4dcb-841c-7e7df107c800 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:44.020516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-db1f32da-37a9-4dcb-841c-7e7df107c800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f136ce3fa644465abcbace626da90de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:44.034416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-db1f32da-37a9-4dcb-841c-7e7df107c800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f136ce3fa644465abcbace626da90de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:44.035158 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-db1f32da-37a9-4dcb-841c-7e7df107c800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f136ce3fa644465abcbace626da90de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:44.125424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-db1f32da-37a9-4dcb-841c-7e7df107c800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f136ce3fa644465abcbace626da90de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:44.126043 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-db1f32da-37a9-4dcb-841c-7e7df107c800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f136ce3fa644465abcbace626da90de'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:44.139404 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1308/2586] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:43 2022] DELETE /identity/v3/users/3f136ce3fa644465abcbace626da90de => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:44.640054 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:44.641631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:44.655664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6cdba-8cf0-40cd-9300-173cf49ab61d admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:44.719693 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378b9bd-3ebd-4ee2-8890-83e52d3c5f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:44.720320 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378b9bd-3ebd-4ee2-8890-83e52d3c5f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:44.720591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378b9bd-3ebd-4ee2-8890-83e52d3c5f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:44.720856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378b9bd-3ebd-4ee2-8890-83e52d3c5f6c admin admin] PATH_INFO: `/v3/projects/9832efd6a2f54c0bad422d42575416eb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:44.721986 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2378b9bd-3ebd-4ee2-8890-83e52d3c5f6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9832efd6a2f54c0bad422d42575416eb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:44.723969 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2378b9bd-3ebd-4ee2-8890-83e52d3c5f6c admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:44.877960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-2378b9bd-3ebd-4ee2-8890-83e52d3c5f6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9832efd6a2f54c0bad422d42575416eb', 'request_id': 'req-2378b9bd-3ebd-4ee2-8890-83e52d3c5f6c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:44.884909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-2378b9bd-3ebd-4ee2-8890-83e52d3c5f6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9832efd6a2f54c0bad422d42575416eb', 'request_id': 'req-2378b9bd-3ebd-4ee2-8890-83e52d3c5f6c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:44.960720 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-2378b9bd-3ebd-4ee2-8890-83e52d3c5f6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9832efd6a2f54c0bad422d42575416eb', 'request_id': 'req-2378b9bd-3ebd-4ee2-8890-83e52d3c5f6c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:44.973151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-2378b9bd-3ebd-4ee2-8890-83e52d3c5f6c admin admin] The token cache is being invalidate because project 9832efd6a2f54c0bad422d42575416eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:44.973443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-2378b9bd-3ebd-4ee2-8890-83e52d3c5f6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:44.974086 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-2378b9bd-3ebd-4ee2-8890-83e52d3c5f6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:44.976080 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1279/2587] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:44 2022] DELETE /identity/v3/projects/9832efd6a2f54c0bad422d42575416eb => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:24:44.980921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631580a-4146-4c2d-8b5e-bad7a8658e00 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:44.981379 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631580a-4146-4c2d-8b5e-bad7a8658e00 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:44.981473 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631580a-4146-4c2d-8b5e-bad7a8658e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:45.001508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3631580a-4146-4c2d-8b5e-bad7a8658e00 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:45.002704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3631580a-4146-4c2d-8b5e-bad7a8658e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:45.066879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1309/2588] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:45.075720 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4224c-13fc-4802-9731-569b67345866 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:45.076000 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4224c-13fc-4802-9731-569b67345866 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:45.076295 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4224c-13fc-4802-9731-569b67345866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:45.095920 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-8ef4224c-13fc-4802-9731-569b67345866 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:45.101186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-8ef4224c-13fc-4802-9731-569b67345866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:45.105222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1280/2589] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:45.109447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631580a-4146-4c2d-8b5e-bad7a8658e00 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:45.119750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631580a-4146-4c2d-8b5e-bad7a8658e00 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:45.124343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23bf6a-e0ec-489f-ac9b-889252e6ae0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:45.124942 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23bf6a-e0ec-489f-ac9b-889252e6ae0b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:45.125222 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23bf6a-e0ec-489f-ac9b-889252e6ae0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:45.125496 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23bf6a-e0ec-489f-ac9b-889252e6ae0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:45.126147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23bf6a-e0ec-489f-ac9b-889252e6ae0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:24:45.126449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23bf6a-e0ec-489f-ac9b-889252e6ae0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:45.128086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23bf6a-e0ec-489f-ac9b-889252e6ae0b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:45.140413 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1310/2590] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:24:45.144790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4224c-13fc-4802-9731-569b67345866 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:45.154633 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4224c-13fc-4802-9731-569b67345866 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:45.159342 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3cd7a8-8567-4bd7-af43-e2487ac3c38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:45.159931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3cd7a8-8567-4bd7-af43-e2487ac3c38a admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:45.160202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3cd7a8-8567-4bd7-af43-e2487ac3c38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:45.160466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3cd7a8-8567-4bd7-af43-e2487ac3c38a admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:45.161097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3cd7a8-8567-4bd7-af43-e2487ac3c38a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:45.163338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3cd7a8-8567-4bd7-af43-e2487ac3c38a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:45.244971 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1281/2591] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:45 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:45.249257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23bf6a-e0ec-489f-ac9b-889252e6ae0b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:45.260098 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23bf6a-e0ec-489f-ac9b-889252e6ae0b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:45.304817 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:45.316420 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536b7c5-3943-47b3-b95e-65b93a270917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:45.317015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536b7c5-3943-47b3-b95e-65b93a270917 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:45.317288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536b7c5-3943-47b3-b95e-65b93a270917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:45.317557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536b7c5-3943-47b3-b95e-65b93a270917 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:45.318190 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536b7c5-3943-47b3-b95e-65b93a270917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:45.320886 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536b7c5-3943-47b3-b95e-65b93a270917 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:45.446980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1311/2592] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:45 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:24:45.450273 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3cd7a8-8567-4bd7-af43-e2487ac3c38a admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:45.460802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3cd7a8-8567-4bd7-af43-e2487ac3c38a admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:45.465570 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbb3ae-310b-4f89-8100-ca188b0841f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:45.466202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbb3ae-310b-4f89-8100-ca188b0841f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:45.466489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbb3ae-310b-4f89-8100-ca188b0841f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:45.466760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbb3ae-310b-4f89-8100-ca188b0841f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:45.467466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbb3ae-310b-4f89-8100-ca188b0841f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:45.469059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbb3ae-310b-4f89-8100-ca188b0841f0 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:45.479743 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1282/2593] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:45.483831 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536b7c5-3943-47b3-b95e-65b93a270917 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:45.494368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536b7c5-3943-47b3-b95e-65b93a270917 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:45.498957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5eb41-f027-49ef-8e92-8b82a0d1e76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:45.499607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5eb41-f027-49ef-8e92-8b82a0d1e76b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:45.499884 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5eb41-f027-49ef-8e92-8b82a0d1e76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:45.500157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5eb41-f027-49ef-8e92-8b82a0d1e76b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:45.500795 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5eb41-f027-49ef-8e92-8b82a0d1e76b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:45.502396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5eb41-f027-49ef-8e92-8b82a0d1e76b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:45.512870 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1312/2594] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:45.517312 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbb3ae-310b-4f89-8100-ca188b0841f0 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:45.527209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbb3ae-310b-4f89-8100-ca188b0841f0 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:45.532435 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef2cd9-f8c2-4cfb-85f7-732a655ec9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:45.533209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef2cd9-f8c2-4cfb-85f7-732a655ec9a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:45.533540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef2cd9-f8c2-4cfb-85f7-732a655ec9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:45.533832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef2cd9-f8c2-4cfb-85f7-732a655ec9a9 admin admin] PATH_INFO: `/v3/projects/9e0d9f8676464ad0804fe1e6ed9874db/users/4d6f887edb4040ed808abefe7f8984f8/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:45.542521 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:45.551587 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ef2cd9-f8c2-4cfb-85f7-732a655ec9a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0d9f8676464ad0804fe1e6ed9874db, user_id=4d6f887edb4040ed808abefe7f8984f8, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:45.555376 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ef2cd9-f8c2-4cfb-85f7-732a655ec9a9 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:45.628500 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1283/2595] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:45 2022] PUT /identity/v3/projects/9e0d9f8676464ad0804fe1e6ed9874db/users/4d6f887edb4040ed808abefe7f8984f8/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:45.640305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9433e1-84a2-474b-ab9a-6c30d9089621 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:45.645746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9433e1-84a2-474b-ab9a-6c30d9089621 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:45.645746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9433e1-84a2-474b-ab9a-6c30d9089621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:45.668950 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4e9433e1-84a2-474b-ab9a-6c30d9089621 None None] MFA Rules not processed for user `4d6f887edb4040ed808abefe7f8984f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:45.670172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4e9433e1-84a2-474b-ab9a-6c30d9089621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:45.735947 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1313/2596] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:45.741393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef2cd9-f8c2-4cfb-85f7-732a655ec9a9 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:45.801541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef2cd9-f8c2-4cfb-85f7-732a655ec9a9 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:45.807164 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb099e2-63b3-4455-b864-d5a3aef6a4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:45.807768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb099e2-63b3-4455-b864-d5a3aef6a4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:45.808049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb099e2-63b3-4455-b864-d5a3aef6a4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:45.808323 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb099e2-63b3-4455-b864-d5a3aef6a4e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:45.808959 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb099e2-63b3-4455-b864-d5a3aef6a4e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:45.810919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb099e2-63b3-4455-b864-d5a3aef6a4e3 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:45.913684 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1284/2597] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:45 2022] POST /identity/v3/projects => generated 365 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:45.915152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9433e1-84a2-474b-ab9a-6c30d9089621 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:45.926121 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9433e1-84a2-474b-ab9a-6c30d9089621 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:45.986104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf40d9-12ab-45ea-b777-81be701b1f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:45.986744 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf40d9-12ab-45ea-b777-81be701b1f45 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:45.987004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf40d9-12ab-45ea-b777-81be701b1f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:45.987320 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf40d9-12ab-45ea-b777-81be701b1f45 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:45.987949 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf40d9-12ab-45ea-b777-81be701b1f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:45.989855 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf40d9-12ab-45ea-b777-81be701b1f45 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:46.090912 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1314/2598] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:45 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:46.094834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb099e2-63b3-4455-b864-d5a3aef6a4e3 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:46.106604 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb099e2-63b3-4455-b864-d5a3aef6a4e3 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:46.111468 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539eb74-66e5-4cb7-8248-88fd53387f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:46.112064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539eb74-66e5-4cb7-8248-88fd53387f43 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:46.112341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539eb74-66e5-4cb7-8248-88fd53387f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:46.112608 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539eb74-66e5-4cb7-8248-88fd53387f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:46.113250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539eb74-66e5-4cb7-8248-88fd53387f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:46.114850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539eb74-66e5-4cb7-8248-88fd53387f43 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:46.126123 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1285/2599] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:46.133623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf40d9-12ab-45ea-b777-81be701b1f45 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:46.141017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf40d9-12ab-45ea-b777-81be701b1f45 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:46.145811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea7283-7dbe-4af9-86f1-fc6299149f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:46.147072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea7283-7dbe-4af9-86f1-fc6299149f55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:46.147353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea7283-7dbe-4af9-86f1-fc6299149f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:46.147627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea7283-7dbe-4af9-86f1-fc6299149f55 admin admin] PATH_INFO: `/v3/projects/a93022d741c8411abb1075b2eb277c00/users/41ed0024487944a09ddfaeb3c01cf3a0/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:46.164280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea7283-7dbe-4af9-86f1-fc6299149f55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93022d741c8411abb1075b2eb277c00, user_id=41ed0024487944a09ddfaeb3c01cf3a0, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:46.167004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea7283-7dbe-4af9-86f1-fc6299149f55 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:46.239356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1315/2600] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:46 2022] PUT /identity/v3/projects/a93022d741c8411abb1075b2eb277c00/users/41ed0024487944a09ddfaeb3c01cf3a0/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:46.243453 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539eb74-66e5-4cb7-8248-88fd53387f43 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:46.253355 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539eb74-66e5-4cb7-8248-88fd53387f43 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:46.314258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3a1d8-657c-451b-80ec-f2ce8cf8ba67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:46.314854 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3a1d8-657c-451b-80ec-f2ce8cf8ba67 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:46.315158 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3a1d8-657c-451b-80ec-f2ce8cf8ba67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:46.315430 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3a1d8-657c-451b-80ec-f2ce8cf8ba67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:46.316062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3a1d8-657c-451b-80ec-f2ce8cf8ba67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:46.317660 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3a1d8-657c-451b-80ec-f2ce8cf8ba67 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:46.331514 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1286/2601] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:46.332847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea7283-7dbe-4af9-86f1-fc6299149f55 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:46.344778 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea7283-7dbe-4af9-86f1-fc6299149f55 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:46.349470 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67015b1d-b218-4241-abc6-ee5190295df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:46.350675 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67015b1d-b218-4241-abc6-ee5190295df5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:46.351099 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67015b1d-b218-4241-abc6-ee5190295df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:46.351373 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67015b1d-b218-4241-abc6-ee5190295df5 admin admin] PATH_INFO: `/v3/projects/a93022d741c8411abb1075b2eb277c00/users/41ed0024487944a09ddfaeb3c01cf3a0/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:46.353324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67015b1d-b218-4241-abc6-ee5190295df5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93022d741c8411abb1075b2eb277c00, user_id=41ed0024487944a09ddfaeb3c01cf3a0, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:46.356101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67015b1d-b218-4241-abc6-ee5190295df5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:46.367084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1316/2602] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:46 2022] PUT /identity/v3/projects/a93022d741c8411abb1075b2eb277c00/users/41ed0024487944a09ddfaeb3c01cf3a0/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:46.377806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dd75c-0d11-4d91-92ba-7f9144d66977 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:46.378093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dd75c-0d11-4d91-92ba-7f9144d66977 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:46.378375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dd75c-0d11-4d91-92ba-7f9144d66977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:46.411202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-981dd75c-0d11-4d91-92ba-7f9144d66977 None None] MFA Rules not processed for user `41ed0024487944a09ddfaeb3c01cf3a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:46.412431 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-981dd75c-0d11-4d91-92ba-7f9144d66977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:46.453888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:46.480877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1287/2603] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:46.492700 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67015b1d-b218-4241-abc6-ee5190295df5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:46.494353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-67015b1d-b218-4241-abc6-ee5190295df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:46.505359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:46.554651 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67015b1d-b218-4241-abc6-ee5190295df5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:46.558479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2aca5b-84cb-45cf-9e46-5936a241f470 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:46.559108 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aca5b-84cb-45cf-9e46-5936a241f470 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:46.559382 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aca5b-84cb-45cf-9e46-5936a241f470 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:46.559648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aca5b-84cb-45cf-9e46-5936a241f470 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:46.570223 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2aca5b-84cb-45cf-9e46-5936a241f470 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:46.571723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2aca5b-84cb-45cf-9e46-5936a241f470 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:46.580955 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1317/2604] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:46 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:46.790911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dd75c-0d11-4d91-92ba-7f9144d66977 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:46.800465 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dd75c-0d11-4d91-92ba-7f9144d66977 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:46.804642 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb27d8-27fb-4f21-b9e5-022e01cb7d0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:46.805234 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb27d8-27fb-4f21-b9e5-022e01cb7d0d service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:46.805507 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb27d8-27fb-4f21-b9e5-022e01cb7d0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:46.805768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb27d8-27fb-4f21-b9e5-022e01cb7d0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:46.860461 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb27d8-27fb-4f21-b9e5-022e01cb7d0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:46.861884 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb27d8-27fb-4f21-b9e5-022e01cb7d0d service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:46.871426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1288/2605] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:50.776093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2aca5b-84cb-45cf-9e46-5936a241f470 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:50.838916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2aca5b-84cb-45cf-9e46-5936a241f470 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:50.843518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f111c1d-ae85-4d2c-9da7-dc1662ca9968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:50.844134 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f111c1d-ae85-4d2c-9da7-dc1662ca9968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:50.844408 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f111c1d-ae85-4d2c-9da7-dc1662ca9968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:50.844672 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f111c1d-ae85-4d2c-9da7-dc1662ca9968 admin admin] PATH_INFO: `/v3/users/4d6f887edb4040ed808abefe7f8984f8` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:50.845759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f111c1d-ae85-4d2c-9da7-dc1662ca9968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d6f887edb4040ed808abefe7f8984f8)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:50.847728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f111c1d-ae85-4d2c-9da7-dc1662ca9968 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:51.052307 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5f111c1d-ae85-4d2c-9da7-dc1662ca9968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d6f887edb4040ed808abefe7f8984f8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:51.078339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5f111c1d-ae85-4d2c-9da7-dc1662ca9968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d6f887edb4040ed808abefe7f8984f8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:51.079093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5f111c1d-ae85-4d2c-9da7-dc1662ca9968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d6f887edb4040ed808abefe7f8984f8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:51.165381 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5f111c1d-ae85-4d2c-9da7-dc1662ca9968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d6f887edb4040ed808abefe7f8984f8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:51.166017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5f111c1d-ae85-4d2c-9da7-dc1662ca9968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d6f887edb4040ed808abefe7f8984f8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:51.179254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1318/2606] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:50 2022] DELETE /identity/v3/users/4d6f887edb4040ed808abefe7f8984f8 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:51.183831 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580a19a-4590-48a3-8158-2da9d6d3b18b None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:51.184278 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580a19a-4590-48a3-8158-2da9d6d3b18b None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:51.184474 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580a19a-4590-48a3-8158-2da9d6d3b18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:51.204148 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-b580a19a-4590-48a3-8158-2da9d6d3b18b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:51.205336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b580a19a-4590-48a3-8158-2da9d6d3b18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:51.263889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1289/2607] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:51.274695 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f111c1d-ae85-4d2c-9da7-dc1662ca9968 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:51.276172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5f111c1d-ae85-4d2c-9da7-dc1662ca9968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:51.310152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:51.333147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f111c1d-ae85-4d2c-9da7-dc1662ca9968 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:51.344752 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cd841-eb90-4a3c-b8cf-ed9644d98c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:51.345374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cd841-eb90-4a3c-b8cf-ed9644d98c15 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:51.345651 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cd841-eb90-4a3c-b8cf-ed9644d98c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:51.345920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cd841-eb90-4a3c-b8cf-ed9644d98c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:51.357060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cd841-eb90-4a3c-b8cf-ed9644d98c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:51.358479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cd841-eb90-4a3c-b8cf-ed9644d98c15 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:51.367840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1319/2608] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:51.643645 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580a19a-4590-48a3-8158-2da9d6d3b18b None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:51.647077 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b580a19a-4590-48a3-8158-2da9d6d3b18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:51.735680 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580a19a-4590-48a3-8158-2da9d6d3b18b None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:51.742625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198944f-8eab-499a-8640-e61cbe798b7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:51.743520 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198944f-8eab-499a-8640-e61cbe798b7b service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:51.743798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198944f-8eab-499a-8640-e61cbe798b7b service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:51.744058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198944f-8eab-499a-8640-e61cbe798b7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:51.745916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0198944f-8eab-499a-8640-e61cbe798b7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:51.760270 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:51.785348 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cd841-eb90-4a3c-b8cf-ed9644d98c15 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:51.788316 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ce2cd841-eb90-4a3c-b8cf-ed9644d98c15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:51.805401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cd841-eb90-4a3c-b8cf-ed9644d98c15 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:51.813155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:51.813855 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:51.814246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:51.814579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad admin admin] PATH_INFO: `/v3/projects/9e0d9f8676464ad0804fe1e6ed9874db` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:51.815802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e0d9f8676464ad0804fe1e6ed9874db)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:51.817812 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:51.837704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198944f-8eab-499a-8640-e61cbe798b7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:51.839362 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198944f-8eab-499a-8640-e61cbe798b7b service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:51.858768 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1290/2609] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:51.937071 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e0d9f8676464ad0804fe1e6ed9874db', 'request_id': 'req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:51.943905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e0d9f8676464ad0804fe1e6ed9874db', 'request_id': 'req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:52.020964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e0d9f8676464ad0804fe1e6ed9874db', 'request_id': 'req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:52.034498 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad admin admin] The token cache is being invalidate because project 9e0d9f8676464ad0804fe1e6ed9874db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:52.034785 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:52.035471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:52.037757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1320/2610] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:51 2022] DELETE /identity/v3/projects/9e0d9f8676464ad0804fe1e6ed9874db => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:24:52.041836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198944f-8eab-499a-8640-e61cbe798b7b service nova] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:52.052518 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198944f-8eab-499a-8640-e61cbe798b7b service nova] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:52.109219 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6c489-2e67-42b1-8946-941d9218d2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:52.109860 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6c489-2e67-42b1-8946-941d9218d2e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:52.110140 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6c489-2e67-42b1-8946-941d9218d2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:52.110421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6c489-2e67-42b1-8946-941d9218d2e1 admin admin] PATH_INFO: `/v3/users/41ed0024487944a09ddfaeb3c01cf3a0` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:52.111590 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6c489-2e67-42b1-8946-941d9218d2e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41ed0024487944a09ddfaeb3c01cf3a0)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:52.116453 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6c489-2e67-42b1-8946-941d9218d2e1 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:52.337905 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-73d6c489-2e67-42b1-8946-941d9218d2e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41ed0024487944a09ddfaeb3c01cf3a0'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:52.338646 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-73d6c489-2e67-42b1-8946-941d9218d2e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41ed0024487944a09ddfaeb3c01cf3a0'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:52.351905 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-73d6c489-2e67-42b1-8946-941d9218d2e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41ed0024487944a09ddfaeb3c01cf3a0'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:52.426582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-73d6c489-2e67-42b1-8946-941d9218d2e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41ed0024487944a09ddfaeb3c01cf3a0'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:52.427268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-73d6c489-2e67-42b1-8946-941d9218d2e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41ed0024487944a09ddfaeb3c01cf3a0'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:52.441709 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1291/2611] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:52 2022] DELETE /identity/v3/users/41ed0024487944a09ddfaeb3c01cf3a0 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:52.852201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:52.853875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:52.869345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583a364-8ba2-4b3f-8d5a-575da0b6d5ad admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:52.926470 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440113db-d91d-4d9d-a1ea-cd84c8ad8a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:52.927038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440113db-d91d-4d9d-a1ea-cd84c8ad8a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:52.927308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440113db-d91d-4d9d-a1ea-cd84c8ad8a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:52.928179 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440113db-d91d-4d9d-a1ea-cd84c8ad8a26 admin admin] PATH_INFO: `/v3/projects/a93022d741c8411abb1075b2eb277c00` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:52.929410 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440113db-d91d-4d9d-a1ea-cd84c8ad8a26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a93022d741c8411abb1075b2eb277c00)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:52.931349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440113db-d91d-4d9d-a1ea-cd84c8ad8a26 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:53.063013 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-440113db-d91d-4d9d-a1ea-cd84c8ad8a26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a93022d741c8411abb1075b2eb277c00', 'request_id': 'req-440113db-d91d-4d9d-a1ea-cd84c8ad8a26'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:53.069180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-440113db-d91d-4d9d-a1ea-cd84c8ad8a26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a93022d741c8411abb1075b2eb277c00', 'request_id': 'req-440113db-d91d-4d9d-a1ea-cd84c8ad8a26'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:53.072786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:53.176964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-440113db-d91d-4d9d-a1ea-cd84c8ad8a26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a93022d741c8411abb1075b2eb277c00', 'request_id': 'req-440113db-d91d-4d9d-a1ea-cd84c8ad8a26'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:53.189512 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-440113db-d91d-4d9d-a1ea-cd84c8ad8a26 admin admin] The token cache is being invalidate because project a93022d741c8411abb1075b2eb277c00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:53.189800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-440113db-d91d-4d9d-a1ea-cd84c8ad8a26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:53.190464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-440113db-d91d-4d9d-a1ea-cd84c8ad8a26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:53.192053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1321/2612] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:52 2022] DELETE /identity/v3/projects/a93022d741c8411abb1075b2eb277c00 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:53.196939 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4bd0e-4c34-4443-80fa-d9e3e27eb97e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:53.197229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4bd0e-4c34-4443-80fa-d9e3e27eb97e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:53.197507 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4bd0e-4c34-4443-80fa-d9e3e27eb97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:53.205968 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:53.216980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-fff4bd0e-4c34-4443-80fa-d9e3e27eb97e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:53.218188 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-fff4bd0e-4c34-4443-80fa-d9e3e27eb97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:53.277623 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1292/2613] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:53.286640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621b25a-3fd3-47c5-83ad-e40b812a9379 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:53.287588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621b25a-3fd3-47c5-83ad-e40b812a9379 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:53.287909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621b25a-3fd3-47c5-83ad-e40b812a9379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:53.308117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d621b25a-3fd3-47c5-83ad-e40b812a9379 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:53.309321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d621b25a-3fd3-47c5-83ad-e40b812a9379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:53.317166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1322/2614] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:53.321545 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4bd0e-4c34-4443-80fa-d9e3e27eb97e None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:53.331133 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4bd0e-4c34-4443-80fa-d9e3e27eb97e None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:53.336883 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457cec5-c5dd-4272-b1ab-4e786542934d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:53.337473 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457cec5-c5dd-4272-b1ab-4e786542934d admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:53.337749 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457cec5-c5dd-4272-b1ab-4e786542934d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:53.338021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457cec5-c5dd-4272-b1ab-4e786542934d admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:53.338665 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457cec5-c5dd-4272-b1ab-4e786542934d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:24:53.338997 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457cec5-c5dd-4272-b1ab-4e786542934d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:53.340601 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457cec5-c5dd-4272-b1ab-4e786542934d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:53.353918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1293/2615] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:24:53.357927 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621b25a-3fd3-47c5-83ad-e40b812a9379 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:53.367326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621b25a-3fd3-47c5-83ad-e40b812a9379 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:53.372038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b9388-0e41-4a6c-81e2-e2c96d921791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:53.372636 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b9388-0e41-4a6c-81e2-e2c96d921791 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:53.372916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b9388-0e41-4a6c-81e2-e2c96d921791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:53.373188 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b9388-0e41-4a6c-81e2-e2c96d921791 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:53.373816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b9388-0e41-4a6c-81e2-e2c96d921791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:53.375775 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b9388-0e41-4a6c-81e2-e2c96d921791 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:53.469791 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1323/2616] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:53 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:53.475811 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457cec5-c5dd-4272-b1ab-4e786542934d admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:53.486196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457cec5-c5dd-4272-b1ab-4e786542934d admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:53.544447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5db6f-ce39-4558-b200-7b6d9efa835a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:53.545056 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5db6f-ce39-4558-b200-7b6d9efa835a admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:53.545336 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5db6f-ce39-4558-b200-7b6d9efa835a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:53.545609 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5db6f-ce39-4558-b200-7b6d9efa835a admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:53.546261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5db6f-ce39-4558-b200-7b6d9efa835a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:53.548239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5db6f-ce39-4558-b200-7b6d9efa835a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:53.652457 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1294/2617] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:53 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:24:53.653624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b9388-0e41-4a6c-81e2-e2c96d921791 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:53.665283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b9388-0e41-4a6c-81e2-e2c96d921791 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:53.669921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94d99f-ae54-4f1b-9f33-d1aca29ac8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:53.670560 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94d99f-ae54-4f1b-9f33-d1aca29ac8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:53.670816 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94d99f-ae54-4f1b-9f33-d1aca29ac8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:53.671112 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94d99f-ae54-4f1b-9f33-d1aca29ac8d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:53.671753 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94d99f-ae54-4f1b-9f33-d1aca29ac8d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:53.673382 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94d99f-ae54-4f1b-9f33-d1aca29ac8d0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:53.687840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1324/2618] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:53.688603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5db6f-ce39-4558-b200-7b6d9efa835a admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:53.698865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5db6f-ce39-4558-b200-7b6d9efa835a admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:53.703988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abb872-4a14-4387-86ae-d3626cc92d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:53.704606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abb872-4a14-4387-86ae-d3626cc92d6a admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:53.704887 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abb872-4a14-4387-86ae-d3626cc92d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:53.705159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abb872-4a14-4387-86ae-d3626cc92d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:53.705833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abb872-4a14-4387-86ae-d3626cc92d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:53.707492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abb872-4a14-4387-86ae-d3626cc92d6a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:53.721168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1295/2619] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:53.726833 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94d99f-ae54-4f1b-9f33-d1aca29ac8d0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:53.737097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94d99f-ae54-4f1b-9f33-d1aca29ac8d0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:53.742857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffa3ca-072f-4920-8c1a-7009eb67e17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:53.743534 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffa3ca-072f-4920-8c1a-7009eb67e17b admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:53.743808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffa3ca-072f-4920-8c1a-7009eb67e17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:53.744075 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffa3ca-072f-4920-8c1a-7009eb67e17b admin admin] PATH_INFO: `/v3/projects/0285a43821d047be94b5265be8123977/users/7f32659d61b64b43942ac966674d34e6/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:53.759130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffa3ca-072f-4920-8c1a-7009eb67e17b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0285a43821d047be94b5265be8123977, user_id=7f32659d61b64b43942ac966674d34e6, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:53.761824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffa3ca-072f-4920-8c1a-7009eb67e17b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:53.858005 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1325/2620] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:53 2022] PUT /identity/v3/projects/0285a43821d047be94b5265be8123977/users/7f32659d61b64b43942ac966674d34e6/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:53.869703 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f713e-87ce-4a4a-bba7-5bb8cd2b5837 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:53.869985 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f713e-87ce-4a4a-bba7-5bb8cd2b5837 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:53.870257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f713e-87ce-4a4a-bba7-5bb8cd2b5837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:53.900664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-794f713e-87ce-4a4a-bba7-5bb8cd2b5837 None None] MFA Rules not processed for user `7f32659d61b64b43942ac966674d34e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:53.901873 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-794f713e-87ce-4a4a-bba7-5bb8cd2b5837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:53.963321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:53.966929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1296/2621] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:53.971722 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffa3ca-072f-4920-8c1a-7009eb67e17b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:54.037669 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffa3ca-072f-4920-8c1a-7009eb67e17b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:54.042780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184e5db-bedd-4d0b-b621-7724e93faa87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:54.043501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184e5db-bedd-4d0b-b621-7724e93faa87 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:54.046471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184e5db-bedd-4d0b-b621-7724e93faa87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:54.046471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184e5db-bedd-4d0b-b621-7724e93faa87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:54.046471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184e5db-bedd-4d0b-b621-7724e93faa87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:54.046791 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184e5db-bedd-4d0b-b621-7724e93faa87 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:54.137151 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1326/2622] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:53 2022] POST /identity/v3/projects => generated 375 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:54.141462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f713e-87ce-4a4a-bba7-5bb8cd2b5837 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:54.152348 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f713e-87ce-4a4a-bba7-5bb8cd2b5837 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:54.211934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91864e-a82e-48ea-8432-0b150d0596ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:54.212545 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91864e-a82e-48ea-8432-0b150d0596ff admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:54.212820 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91864e-a82e-48ea-8432-0b150d0596ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:54.213090 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91864e-a82e-48ea-8432-0b150d0596ff admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:54.213776 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91864e-a82e-48ea-8432-0b150d0596ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:54.215689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91864e-a82e-48ea-8432-0b150d0596ff admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:54.338248 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1297/2623] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:54 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:54.348574 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184e5db-bedd-4d0b-b621-7724e93faa87 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:54.371155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184e5db-bedd-4d0b-b621-7724e93faa87 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:54.371155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0a131-5ee7-43d4-8d67-ab0826751211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:54.371155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a131-5ee7-43d4-8d67-ab0826751211 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:54.371155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a131-5ee7-43d4-8d67-ab0826751211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:54.374499 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a131-5ee7-43d4-8d67-ab0826751211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:54.374499 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0a131-5ee7-43d4-8d67-ab0826751211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:54.374499 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0a131-5ee7-43d4-8d67-ab0826751211 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:54.383945 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1327/2624] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:54.388341 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91864e-a82e-48ea-8432-0b150d0596ff admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:54.399229 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91864e-a82e-48ea-8432-0b150d0596ff admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:54.404171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19f88e-ebd6-49ef-bed3-ef79512d4400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:54.404824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19f88e-ebd6-49ef-bed3-ef79512d4400 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:54.405641 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19f88e-ebd6-49ef-bed3-ef79512d4400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:54.405641 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19f88e-ebd6-49ef-bed3-ef79512d4400 admin admin] PATH_INFO: `/v3/projects/3bdfd96f67454aed909b7b8d317c89b9/users/39b8d7d194ac4f2b96eb359456890fef/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:54.422163 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19f88e-ebd6-49ef-bed3-ef79512d4400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bdfd96f67454aed909b7b8d317c89b9, user_id=39b8d7d194ac4f2b96eb359456890fef, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:54.425055 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19f88e-ebd6-49ef-bed3-ef79512d4400 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:54.508718 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1298/2625] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:54 2022] PUT /identity/v3/projects/3bdfd96f67454aed909b7b8d317c89b9/users/39b8d7d194ac4f2b96eb359456890fef/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:54.512743 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0a131-5ee7-43d4-8d67-ab0826751211 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:54.523318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0a131-5ee7-43d4-8d67-ab0826751211 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:54.580443 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:54.587120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230c21a-f404-4246-9aa6-ddff4ba4c521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:54.587736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230c21a-f404-4246-9aa6-ddff4ba4c521 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:54.588500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230c21a-f404-4246-9aa6-ddff4ba4c521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:54.588500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230c21a-f404-4246-9aa6-ddff4ba4c521 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:54.588987 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230c21a-f404-4246-9aa6-ddff4ba4c521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:54.590657 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230c21a-f404-4246-9aa6-ddff4ba4c521 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:54.602715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1328/2626] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:24:54.612107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19f88e-ebd6-49ef-bed3-ef79512d4400 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:54.625184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19f88e-ebd6-49ef-bed3-ef79512d4400 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:54.630067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214640c-88c2-4b70-a84f-32a6c8cfe350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:54.631426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214640c-88c2-4b70-a84f-32a6c8cfe350 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:54.631714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214640c-88c2-4b70-a84f-32a6c8cfe350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:54.631991 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214640c-88c2-4b70-a84f-32a6c8cfe350 admin admin] PATH_INFO: `/v3/projects/3bdfd96f67454aed909b7b8d317c89b9/users/39b8d7d194ac4f2b96eb359456890fef/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:54.634226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214640c-88c2-4b70-a84f-32a6c8cfe350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bdfd96f67454aed909b7b8d317c89b9, user_id=39b8d7d194ac4f2b96eb359456890fef, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:54.636933 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214640c-88c2-4b70-a84f-32a6c8cfe350 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:54.648905 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1299/2627] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:24:54 2022] PUT /identity/v3/projects/3bdfd96f67454aed909b7b8d317c89b9/users/39b8d7d194ac4f2b96eb359456890fef/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:24:54.659676 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51a2a8-b1d7-4580-9b25-e97c5980c443 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:54.659960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51a2a8-b1d7-4580-9b25-e97c5980c443 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:54.660317 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51a2a8-b1d7-4580-9b25-e97c5980c443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:54.692103 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-af51a2a8-b1d7-4580-9b25-e97c5980c443 None None] MFA Rules not processed for user `39b8d7d194ac4f2b96eb359456890fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:54.693810 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-af51a2a8-b1d7-4580-9b25-e97c5980c443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:54.779298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1329/2628] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:24:54.788010 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214640c-88c2-4b70-a84f-32a6c8cfe350 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:54.789806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0214640c-88c2-4b70-a84f-32a6c8cfe350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:54.854798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214640c-88c2-4b70-a84f-32a6c8cfe350 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:54.858552 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0b76b-eefd-4798-b453-58d914af846d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:54.863411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0b76b-eefd-4798-b453-58d914af846d service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:54.863411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0b76b-eefd-4798-b453-58d914af846d service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:54.863411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0b76b-eefd-4798-b453-58d914af846d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:54.921290 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0b76b-eefd-4798-b453-58d914af846d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:54.922761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0b76b-eefd-4798-b453-58d914af846d service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:54.932566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1300/2629] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:55.607640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51a2a8-b1d7-4580-9b25-e97c5980c443 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:55.609167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-af51a2a8-b1d7-4580-9b25-e97c5980c443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:55.706061 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51a2a8-b1d7-4580-9b25-e97c5980c443 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:55.713722 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20877e2c-cadc-4052-aeef-86335b0996cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:55.714591 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20877e2c-cadc-4052-aeef-86335b0996cc service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:55.714958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20877e2c-cadc-4052-aeef-86335b0996cc service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:55.715308 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20877e2c-cadc-4052-aeef-86335b0996cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:55.717358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-20877e2c-cadc-4052-aeef-86335b0996cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:55.736965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20877e2c-cadc-4052-aeef-86335b0996cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:55.737416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20877e2c-cadc-4052-aeef-86335b0996cc service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:55.763293 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1330/2630] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:56.425791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0b76b-eefd-4798-b453-58d914af846d service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:56.427956 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-dbb0b76b-eefd-4798-b453-58d914af846d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:56.440111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:56.518545 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0b76b-eefd-4798-b453-58d914af846d service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:56.523837 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364674d0-b679-4d5f-b065-4d846d75fe3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6074872303d3468fb566378fa34e8bea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:56.524450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364674d0-b679-4d5f-b065-4d846d75fe3a service placement] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:56.524731 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364674d0-b679-4d5f-b065-4d846d75fe3a service placement] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:56.525005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364674d0-b679-4d5f-b065-4d846d75fe3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:56.526734 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-364674d0-b679-4d5f-b065-4d846d75fe3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:56.539538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364674d0-b679-4d5f-b065-4d846d75fe3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:56.541359 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364674d0-b679-4d5f-b065-4d846d75fe3a service placement] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:56.555199 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1301/2631] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:24:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:24:57.274512 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20877e2c-cadc-4052-aeef-86335b0996cc service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:57.276160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-20877e2c-cadc-4052-aeef-86335b0996cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:57.347537 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20877e2c-cadc-4052-aeef-86335b0996cc service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:57.353208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67cdb5-0df5-4531-a856-2482459ad534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:57.354189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67cdb5-0df5-4531-a856-2482459ad534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:57.354189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67cdb5-0df5-4531-a856-2482459ad534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:57.354446 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67cdb5-0df5-4531-a856-2482459ad534 admin admin] PATH_INFO: `/v3/users/b3835d1937e1459a8981bb5a5bc16281` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:57.355503 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67cdb5-0df5-4531-a856-2482459ad534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3835d1937e1459a8981bb5a5bc16281)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:57.357433 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67cdb5-0df5-4531-a856-2482459ad534 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:57.361991 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:57.757041 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a67cdb5-0df5-4531-a856-2482459ad534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3835d1937e1459a8981bb5a5bc16281'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:57.777246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a67cdb5-0df5-4531-a856-2482459ad534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3835d1937e1459a8981bb5a5bc16281'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:57.777984 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a67cdb5-0df5-4531-a856-2482459ad534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3835d1937e1459a8981bb5a5bc16281'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:57.891124 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a67cdb5-0df5-4531-a856-2482459ad534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3835d1937e1459a8981bb5a5bc16281'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:57.893463 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a67cdb5-0df5-4531-a856-2482459ad534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3835d1937e1459a8981bb5a5bc16281'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:57.905329 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1331/2632] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:57 2022] DELETE /identity/v3/users/b3835d1937e1459a8981bb5a5bc16281 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:58.148896 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364674d0-b679-4d5f-b065-4d846d75fe3a service placement] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:58.160805 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364674d0-b679-4d5f-b065-4d846d75fe3a service placement] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:58.224817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b9597-1abe-401d-8067-bccbc1ef98a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:58.226750 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b9597-1abe-401d-8067-bccbc1ef98a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:58.226750 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b9597-1abe-401d-8067-bccbc1ef98a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:58.226750 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b9597-1abe-401d-8067-bccbc1ef98a9 admin admin] PATH_INFO: `/v3/projects/f171e79b21624734a13652fb85475ab7` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:58.227379 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b9597-1abe-401d-8067-bccbc1ef98a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f171e79b21624734a13652fb85475ab7)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:58.229285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b9597-1abe-401d-8067-bccbc1ef98a9 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:58.400718 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-362b9597-1abe-401d-8067-bccbc1ef98a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f171e79b21624734a13652fb85475ab7', 'request_id': 'req-362b9597-1abe-401d-8067-bccbc1ef98a9'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:58.406855 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-362b9597-1abe-401d-8067-bccbc1ef98a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f171e79b21624734a13652fb85475ab7', 'request_id': 'req-362b9597-1abe-401d-8067-bccbc1ef98a9'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:58.559590 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-362b9597-1abe-401d-8067-bccbc1ef98a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f171e79b21624734a13652fb85475ab7', 'request_id': 'req-362b9597-1abe-401d-8067-bccbc1ef98a9'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:58.570635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-362b9597-1abe-401d-8067-bccbc1ef98a9 admin admin] The token cache is being invalidate because project f171e79b21624734a13652fb85475ab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:58.571014 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-362b9597-1abe-401d-8067-bccbc1ef98a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:58.571649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-362b9597-1abe-401d-8067-bccbc1ef98a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:58.573168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1302/2633] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:58 2022] DELETE /identity/v3/projects/f171e79b21624734a13652fb85475ab7 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:58.577390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67cdb5-0df5-4531-a856-2482459ad534 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:58.578879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5a67cdb5-0df5-4531-a856-2482459ad534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:58.595826 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67cdb5-0df5-4531-a856-2482459ad534 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:58.659823 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8795f1-d8cf-404b-9dba-606461f5b578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:58.660480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8795f1-d8cf-404b-9dba-606461f5b578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:58.660750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8795f1-d8cf-404b-9dba-606461f5b578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:58.661014 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8795f1-d8cf-404b-9dba-606461f5b578 admin admin] PATH_INFO: `/v3/users/9f3ad007b3d2495a892ea7c236f0c3fc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:58.662834 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8795f1-d8cf-404b-9dba-606461f5b578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f3ad007b3d2495a892ea7c236f0c3fc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:58.664813 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8795f1-d8cf-404b-9dba-606461f5b578 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:58.863105 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-db8795f1-d8cf-404b-9dba-606461f5b578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f3ad007b3d2495a892ea7c236f0c3fc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:58.875057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-db8795f1-d8cf-404b-9dba-606461f5b578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f3ad007b3d2495a892ea7c236f0c3fc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:58.875729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-db8795f1-d8cf-404b-9dba-606461f5b578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f3ad007b3d2495a892ea7c236f0c3fc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:58.970516 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-db8795f1-d8cf-404b-9dba-606461f5b578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f3ad007b3d2495a892ea7c236f0c3fc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:58.971167 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-db8795f1-d8cf-404b-9dba-606461f5b578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f3ad007b3d2495a892ea7c236f0c3fc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:58.983683 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1332/2634] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:24:58 2022] DELETE /identity/v3/users/9f3ad007b3d2495a892ea7c236f0c3fc => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:59.252356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b9597-1abe-401d-8067-bccbc1ef98a9 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:59.253816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-362b9597-1abe-401d-8067-bccbc1ef98a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:59.267773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b9597-1abe-401d-8067-bccbc1ef98a9 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:59.293263 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:59.326914 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1733dce-f673-4a18-bc0c-1f4a5677a9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:59.327576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1733dce-f673-4a18-bc0c-1f4a5677a9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:59.327858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1733dce-f673-4a18-bc0c-1f4a5677a9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:59.328130 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1733dce-f673-4a18-bc0c-1f4a5677a9ec admin admin] PATH_INFO: `/v3/projects/81d49bb855fa4f0cb2fae0cbcc2331f3` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:59.329308 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1733dce-f673-4a18-bc0c-1f4a5677a9ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=81d49bb855fa4f0cb2fae0cbcc2331f3)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:59.331282 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1733dce-f673-4a18-bc0c-1f4a5677a9ec admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:59.480171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e1733dce-f673-4a18-bc0c-1f4a5677a9ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81d49bb855fa4f0cb2fae0cbcc2331f3', 'request_id': 'req-e1733dce-f673-4a18-bc0c-1f4a5677a9ec'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:59.486230 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e1733dce-f673-4a18-bc0c-1f4a5677a9ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81d49bb855fa4f0cb2fae0cbcc2331f3', 'request_id': 'req-e1733dce-f673-4a18-bc0c-1f4a5677a9ec'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:59.564876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e1733dce-f673-4a18-bc0c-1f4a5677a9ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81d49bb855fa4f0cb2fae0cbcc2331f3', 'request_id': 'req-e1733dce-f673-4a18-bc0c-1f4a5677a9ec'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:59.576058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e1733dce-f673-4a18-bc0c-1f4a5677a9ec admin admin] The token cache is being invalidate because project 81d49bb855fa4f0cb2fae0cbcc2331f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:24:59.586363 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e1733dce-f673-4a18-bc0c-1f4a5677a9ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:59.586363 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e1733dce-f673-4a18-bc0c-1f4a5677a9ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:24:59.586363 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1303/2635] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:24:59 2022] DELETE /identity/v3/projects/81d49bb855fa4f0cb2fae0cbcc2331f3 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:24:59.586754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aeac34-9298-45ef-b555-22045736f91d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:59.586754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aeac34-9298-45ef-b555-22045736f91d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:59.586754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aeac34-9298-45ef-b555-22045736f91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:59.591656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:24:59.605084 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-82aeac34-9298-45ef-b555-22045736f91d None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:59.606270 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-82aeac34-9298-45ef-b555-22045736f91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:59.664871 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1333/2636] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:59.672388 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3716f6-85c2-413b-a6b1-917cb4bed94d None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:59.672665 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3716f6-85c2-413b-a6b1-917cb4bed94d None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:59.672933 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3716f6-85c2-413b-a6b1-917cb4bed94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:59.693364 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-1f3716f6-85c2-413b-a6b1-917cb4bed94d None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:24:59.694557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1f3716f6-85c2-413b-a6b1-917cb4bed94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:24:59.703672 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1304/2637] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:24:59.707841 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aeac34-9298-45ef-b555-22045736f91d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:59.717267 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aeac34-9298-45ef-b555-22045736f91d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:59.721843 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66baed9b-ed6a-4c90-b33f-4df8b55a067c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:59.722434 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baed9b-ed6a-4c90-b33f-4df8b55a067c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:59.722704 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baed9b-ed6a-4c90-b33f-4df8b55a067c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:59.722988 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66baed9b-ed6a-4c90-b33f-4df8b55a067c admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:59.727366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66baed9b-ed6a-4c90-b33f-4df8b55a067c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:24:59.727366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66baed9b-ed6a-4c90-b33f-4df8b55a067c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:59.727366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66baed9b-ed6a-4c90-b33f-4df8b55a067c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:59.741609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1334/2638] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:24:59.746037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3716f6-85c2-413b-a6b1-917cb4bed94d None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:59.755280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3716f6-85c2-413b-a6b1-917cb4bed94d None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:59.759871 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052f16d-fd8d-429d-a3e7-ec8b13cd7663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:59.760454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052f16d-fd8d-429d-a3e7-ec8b13cd7663 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:59.760781 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052f16d-fd8d-429d-a3e7-ec8b13cd7663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:59.761053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052f16d-fd8d-429d-a3e7-ec8b13cd7663 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:59.761690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052f16d-fd8d-429d-a3e7-ec8b13cd7663 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:59.763626 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052f16d-fd8d-429d-a3e7-ec8b13cd7663 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:59.846266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1305/2639] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:24:59 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:59.851010 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66baed9b-ed6a-4c90-b33f-4df8b55a067c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:24:59.860829 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66baed9b-ed6a-4c90-b33f-4df8b55a067c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:24:59.916626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a6c3c-6fa1-47c9-bbbd-8963b79cbe4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:24:59.917229 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a6c3c-6fa1-47c9-bbbd-8963b79cbe4e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:24:59.917505 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a6c3c-6fa1-47c9-bbbd-8963b79cbe4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:24:59.917775 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a6c3c-6fa1-47c9-bbbd-8963b79cbe4e admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:24:59.918401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a6c3c-6fa1-47c9-bbbd-8963b79cbe4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:24:59.920362 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a6c3c-6fa1-47c9-bbbd-8963b79cbe4e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:24:59.991754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1335/2640] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:24:59 2022] POST /identity/v3/users => generated 287 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:24:59.995981 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052f16d-fd8d-429d-a3e7-ec8b13cd7663 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:00.006677 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052f16d-fd8d-429d-a3e7-ec8b13cd7663 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:00.011385 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77125ed-81b9-4253-8c9e-7e3ba9205168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:00.013880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77125ed-81b9-4253-8c9e-7e3ba9205168 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:00.013880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77125ed-81b9-4253-8c9e-7e3ba9205168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:00.013880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77125ed-81b9-4253-8c9e-7e3ba9205168 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:00.013880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77125ed-81b9-4253-8c9e-7e3ba9205168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:00.014961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77125ed-81b9-4253-8c9e-7e3ba9205168 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:00.025402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1306/2641] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:25:00.029365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a6c3c-6fa1-47c9-bbbd-8963b79cbe4e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:00.039269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a6c3c-6fa1-47c9-bbbd-8963b79cbe4e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:00.043791 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c0152-3aff-4e03-a972-27df1d836f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:00.044369 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c0152-3aff-4e03-a972-27df1d836f70 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:00.044642 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c0152-3aff-4e03-a972-27df1d836f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:00.044907 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c0152-3aff-4e03-a972-27df1d836f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:00.045539 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c0152-3aff-4e03-a972-27df1d836f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:00.047123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c0152-3aff-4e03-a972-27df1d836f70 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:00.057898 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1336/2642] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:25:00.061903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77125ed-81b9-4253-8c9e-7e3ba9205168 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:00.072820 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77125ed-81b9-4253-8c9e-7e3ba9205168 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:00.077305 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8473da-ca4c-443a-8d05-e307c525bc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:00.077936 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8473da-ca4c-443a-8d05-e307c525bc62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:00.078209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8473da-ca4c-443a-8d05-e307c525bc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:00.078475 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8473da-ca4c-443a-8d05-e307c525bc62 admin admin] PATH_INFO: `/v3/projects/865003d20a044dd59b2e2a13362da6e1/users/1c6b6f97f49a49b09cb3f2e96c4a39a9/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:00.092529 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8473da-ca4c-443a-8d05-e307c525bc62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=865003d20a044dd59b2e2a13362da6e1, user_id=1c6b6f97f49a49b09cb3f2e96c4a39a9, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:00.095194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8473da-ca4c-443a-8d05-e307c525bc62 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:00.160467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1307/2643] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:25:00 2022] PUT /identity/v3/projects/865003d20a044dd59b2e2a13362da6e1/users/1c6b6f97f49a49b09cb3f2e96c4a39a9/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:00.170736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118cc0f-9d84-4c36-b170-31ab2c38ba93 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:00.171060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118cc0f-9d84-4c36-b170-31ab2c38ba93 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:00.171336 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118cc0f-9d84-4c36-b170-31ab2c38ba93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:00.200849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e118cc0f-9d84-4c36-b170-31ab2c38ba93 None None] MFA Rules not processed for user `1c6b6f97f49a49b09cb3f2e96c4a39a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:00.202058 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e118cc0f-9d84-4c36-b170-31ab2c38ba93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:00.261161 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:00.266393 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1337/2644] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:00.283910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8473da-ca4c-443a-8d05-e307c525bc62 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:00.285757 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7d8473da-ca4c-443a-8d05-e307c525bc62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:00.346019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8473da-ca4c-443a-8d05-e307c525bc62 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:00.350199 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942961e-3d5f-46c8-8bcb-efb28b701c26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:00.350797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942961e-3d5f-46c8-8bcb-efb28b701c26 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:00.351184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942961e-3d5f-46c8-8bcb-efb28b701c26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:00.351501 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942961e-3d5f-46c8-8bcb-efb28b701c26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:00.357641 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:00.359795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942961e-3d5f-46c8-8bcb-efb28b701c26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:00.361224 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942961e-3d5f-46c8-8bcb-efb28b701c26 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:00.371085 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1308/2645] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:25:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:25:00.394931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118cc0f-9d84-4c36-b170-31ab2c38ba93 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:00.458670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118cc0f-9d84-4c36-b170-31ab2c38ba93 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:00.463774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49cd59-efa8-48e2-a115-20666ca19ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:00.464395 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49cd59-efa8-48e2-a115-20666ca19ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:00.466396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49cd59-efa8-48e2-a115-20666ca19ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:00.466396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49cd59-efa8-48e2-a115-20666ca19ba2 admin admin] PATH_INFO: `/v3/users/1c6b6f97f49a49b09cb3f2e96c4a39a9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:00.466396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49cd59-efa8-48e2-a115-20666ca19ba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c6b6f97f49a49b09cb3f2e96c4a39a9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:00.468284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49cd59-efa8-48e2-a115-20666ca19ba2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:00.669828 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7a49cd59-efa8-48e2-a115-20666ca19ba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c6b6f97f49a49b09cb3f2e96c4a39a9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:00.682502 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7a49cd59-efa8-48e2-a115-20666ca19ba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c6b6f97f49a49b09cb3f2e96c4a39a9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:00.683220 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7a49cd59-efa8-48e2-a115-20666ca19ba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c6b6f97f49a49b09cb3f2e96c4a39a9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:00.754242 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7a49cd59-efa8-48e2-a115-20666ca19ba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c6b6f97f49a49b09cb3f2e96c4a39a9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:00.754926 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7a49cd59-efa8-48e2-a115-20666ca19ba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c6b6f97f49a49b09cb3f2e96c4a39a9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:00.771912 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1338/2646] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:25:00 2022] DELETE /identity/v3/users/1c6b6f97f49a49b09cb3f2e96c4a39a9 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:00.778164 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc138cff-8b20-4662-8808-6070629883bf None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:00.778445 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc138cff-8b20-4662-8808-6070629883bf None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:00.778716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc138cff-8b20-4662-8808-6070629883bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:00.797751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-fc138cff-8b20-4662-8808-6070629883bf None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:00.798924 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-fc138cff-8b20-4662-8808-6070629883bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:00.856525 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1309/2647] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:00.866176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49cd59-efa8-48e2-a115-20666ca19ba2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:00.867601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7a49cd59-efa8-48e2-a115-20666ca19ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:00.924948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49cd59-efa8-48e2-a115-20666ca19ba2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:00.928805 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83912f-e49f-4feb-9e0f-4eecc7550570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:00.929396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83912f-e49f-4feb-9e0f-4eecc7550570 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:00.929668 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83912f-e49f-4feb-9e0f-4eecc7550570 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:00.929936 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83912f-e49f-4feb-9e0f-4eecc7550570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:00.938622 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83912f-e49f-4feb-9e0f-4eecc7550570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:00.940061 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83912f-e49f-4feb-9e0f-4eecc7550570 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:00.949488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1339/2648] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:25:01.429312 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc138cff-8b20-4662-8808-6070629883bf None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:01.430810 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-fc138cff-8b20-4662-8808-6070629883bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:01.444309 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc138cff-8b20-4662-8808-6070629883bf None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:01.449013 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd65a37-1a32-49ea-b27e-f57d41ad7848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:01.449654 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd65a37-1a32-49ea-b27e-f57d41ad7848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:01.450068 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd65a37-1a32-49ea-b27e-f57d41ad7848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:01.450352 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd65a37-1a32-49ea-b27e-f57d41ad7848 admin admin] PATH_INFO: `/v3/projects/865003d20a044dd59b2e2a13362da6e1` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:01.451512 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd65a37-1a32-49ea-b27e-f57d41ad7848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=865003d20a044dd59b2e2a13362da6e1)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:01.453496 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd65a37-1a32-49ea-b27e-f57d41ad7848 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:01.587812 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-5fd65a37-1a32-49ea-b27e-f57d41ad7848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '865003d20a044dd59b2e2a13362da6e1', 'request_id': 'req-5fd65a37-1a32-49ea-b27e-f57d41ad7848'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:01.594508 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-5fd65a37-1a32-49ea-b27e-f57d41ad7848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '865003d20a044dd59b2e2a13362da6e1', 'request_id': 'req-5fd65a37-1a32-49ea-b27e-f57d41ad7848'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:01.673834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-5fd65a37-1a32-49ea-b27e-f57d41ad7848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '865003d20a044dd59b2e2a13362da6e1', 'request_id': 'req-5fd65a37-1a32-49ea-b27e-f57d41ad7848'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:01.685769 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-5fd65a37-1a32-49ea-b27e-f57d41ad7848 admin admin] The token cache is being invalidate because project 865003d20a044dd59b2e2a13362da6e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:25:01.686059 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-5fd65a37-1a32-49ea-b27e-f57d41ad7848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:01.686750 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-5fd65a37-1a32-49ea-b27e-f57d41ad7848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:01.688289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1310/2649] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:25:01 2022] DELETE /identity/v3/projects/865003d20a044dd59b2e2a13362da6e1 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:01.693934 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645ff78-cea6-42ba-b313-8fae8d741a43 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:01.694403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645ff78-cea6-42ba-b313-8fae8d741a43 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:01.694527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645ff78-cea6-42ba-b313-8fae8d741a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:01.714055 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2645ff78-cea6-42ba-b313-8fae8d741a43 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:01.715297 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2645ff78-cea6-42ba-b313-8fae8d741a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:01.728424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:01.778754 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1340/2650] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:01.788721 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07492c85-efc8-471d-92e8-875049eae6bc None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:01.788925 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07492c85-efc8-471d-92e8-875049eae6bc None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:01.789181 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07492c85-efc8-471d-92e8-875049eae6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:01.811066 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-07492c85-efc8-471d-92e8-875049eae6bc None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:01.812362 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-07492c85-efc8-471d-92e8-875049eae6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:01.822140 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1311/2651] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:01.826225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645ff78-cea6-42ba-b313-8fae8d741a43 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:01.836182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645ff78-cea6-42ba-b313-8fae8d741a43 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:01.840762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86cde0-3f59-444a-91aa-33140aef9062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:01.841384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86cde0-3f59-444a-91aa-33140aef9062 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:01.841661 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86cde0-3f59-444a-91aa-33140aef9062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:01.841927 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86cde0-3f59-444a-91aa-33140aef9062 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:01.842570 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86cde0-3f59-444a-91aa-33140aef9062 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:25:01.842867 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86cde0-3f59-444a-91aa-33140aef9062 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:01.844493 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86cde0-3f59-444a-91aa-33140aef9062 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:01.856567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1341/2652] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 08:25:01.860805 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07492c85-efc8-471d-92e8-875049eae6bc None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:01.871099 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07492c85-efc8-471d-92e8-875049eae6bc None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:01.875869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa4426-14e9-4daf-a039-d61a24c49d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:01.876460 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa4426-14e9-4daf-a039-d61a24c49d55 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:01.876737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa4426-14e9-4daf-a039-d61a24c49d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:01.877005 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa4426-14e9-4daf-a039-d61a24c49d55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:01.877630 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa4426-14e9-4daf-a039-d61a24c49d55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:01.879578 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa4426-14e9-4daf-a039-d61a24c49d55 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:01.972440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1312/2653] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:25:01 2022] POST /identity/v3/projects => generated 391 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:25:01.977334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86cde0-3f59-444a-91aa-33140aef9062 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:01.987297 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86cde0-3f59-444a-91aa-33140aef9062 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:02.045294 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa724690-16a2-483a-910f-e6b499ea1f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:02.045890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa724690-16a2-483a-910f-e6b499ea1f73 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:02.046164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa724690-16a2-483a-910f-e6b499ea1f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:02.046427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa724690-16a2-483a-910f-e6b499ea1f73 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:02.047078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa724690-16a2-483a-910f-e6b499ea1f73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:02.048986 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa724690-16a2-483a-910f-e6b499ea1f73 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:02.152413 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1342/2654] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:25:01 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:25:02.153938 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa4426-14e9-4daf-a039-d61a24c49d55 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:02.164909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa4426-14e9-4daf-a039-d61a24c49d55 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:02.169779 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e7188-73fa-4c10-b17b-dfa924de3c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:02.170376 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e7188-73fa-4c10-b17b-dfa924de3c6c admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:02.170656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e7188-73fa-4c10-b17b-dfa924de3c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:02.170930 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e7188-73fa-4c10-b17b-dfa924de3c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:02.171624 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e7188-73fa-4c10-b17b-dfa924de3c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:02.173212 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e7188-73fa-4c10-b17b-dfa924de3c6c admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:02.183382 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1313/2655] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:25:02.187441 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa724690-16a2-483a-910f-e6b499ea1f73 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:02.198895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa724690-16a2-483a-910f-e6b499ea1f73 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:02.204176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51136d-0015-4025-8f98-ebfe462c6e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:02.204774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51136d-0015-4025-8f98-ebfe462c6e56 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:02.205045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51136d-0015-4025-8f98-ebfe462c6e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:02.205383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51136d-0015-4025-8f98-ebfe462c6e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:02.206129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51136d-0015-4025-8f98-ebfe462c6e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:02.207942 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51136d-0015-4025-8f98-ebfe462c6e56 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:02.219268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1343/2656] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:25:02.223468 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e7188-73fa-4c10-b17b-dfa924de3c6c admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:02.233708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e7188-73fa-4c10-b17b-dfa924de3c6c admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:02.238596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5989aa9-e60c-40ed-b935-5a01fbb66157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:02.239274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5989aa9-e60c-40ed-b935-5a01fbb66157 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:02.240236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5989aa9-e60c-40ed-b935-5a01fbb66157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:02.240543 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5989aa9-e60c-40ed-b935-5a01fbb66157 admin admin] PATH_INFO: `/v3/projects/c8115e49b70445419388b5746ea2e296/users/2b0b24d52238425494bb2192e3615e56/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:02.256948 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5989aa9-e60c-40ed-b935-5a01fbb66157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8115e49b70445419388b5746ea2e296, user_id=2b0b24d52238425494bb2192e3615e56, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:02.259738 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5989aa9-e60c-40ed-b935-5a01fbb66157 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:02.343816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1314/2657] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:25:02 2022] PUT /identity/v3/projects/c8115e49b70445419388b5746ea2e296/users/2b0b24d52238425494bb2192e3615e56/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:02.354728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e316e-ead8-487a-a48a-155c5922028b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:02.355018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e316e-ead8-487a-a48a-155c5922028b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:02.355288 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e316e-ead8-487a-a48a-155c5922028b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:02.388243 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ea6e316e-ead8-487a-a48a-155c5922028b None None] MFA Rules not processed for user `2b0b24d52238425494bb2192e3615e56`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:02.389950 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ea6e316e-ead8-487a-a48a-155c5922028b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:02.466376 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1344/2658] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:02 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:02.483987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5989aa9-e60c-40ed-b935-5a01fbb66157 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:02.483987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e5989aa9-e60c-40ed-b935-5a01fbb66157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:02.498451 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:02.538490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5989aa9-e60c-40ed-b935-5a01fbb66157 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:02.542233 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7aa3b2-acc6-4dce-8b5c-a3ae57d36703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:02.542832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7aa3b2-acc6-4dce-8b5c-a3ae57d36703 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:02.543252 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7aa3b2-acc6-4dce-8b5c-a3ae57d36703 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:02.543527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7aa3b2-acc6-4dce-8b5c-a3ae57d36703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:02.551617 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7aa3b2-acc6-4dce-8b5c-a3ae57d36703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:02.554130 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7aa3b2-acc6-4dce-8b5c-a3ae57d36703 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:02.562816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1315/2659] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:25:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:25:02.848142 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e316e-ead8-487a-a48a-155c5922028b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:02.850350 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ea6e316e-ead8-487a-a48a-155c5922028b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:02.862586 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e316e-ead8-487a-a48a-155c5922028b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:02.867258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b987fc-c30b-4a89-a733-9e1b1fa04da2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:02.867873 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b987fc-c30b-4a89-a733-9e1b1fa04da2 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:02.868153 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b987fc-c30b-4a89-a733-9e1b1fa04da2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:02.868424 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b987fc-c30b-4a89-a733-9e1b1fa04da2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:02.870206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-00b987fc-c30b-4a89-a733-9e1b1fa04da2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:02.886644 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:02.942449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b987fc-c30b-4a89-a733-9e1b1fa04da2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:02.943931 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b987fc-c30b-4a89-a733-9e1b1fa04da2 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:02.953276 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1345/2660] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:25:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:25:12.639704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=19 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:13.033364 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=23 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:14.333732 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7aa3b2-acc6-4dce-8b5c-a3ae57d36703 service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:14.333732 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-8b7aa3b2-acc6-4dce-8b5c-a3ae57d36703 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:14.394691 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7aa3b2-acc6-4dce-8b5c-a3ae57d36703 service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:14.398501 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616d2ef-f6da-4fbf-b531-315c3b37f42f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:14.399771 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616d2ef-f6da-4fbf-b531-315c3b37f42f service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:14.400052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616d2ef-f6da-4fbf-b531-315c3b37f42f service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:14.400328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616d2ef-f6da-4fbf-b531-315c3b37f42f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:14.402105 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5616d2ef-f6da-4fbf-b531-315c3b37f42f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:14.479133 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5616d2ef-f6da-4fbf-b531-315c3b37f42f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:14.480581 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5616d2ef-f6da-4fbf-b531-315c3b37f42f service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:14.490356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1316/2661] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:25:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:25:18.063269 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b987fc-c30b-4a89-a733-9e1b1fa04da2 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:18.064724 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-00b987fc-c30b-4a89-a733-9e1b1fa04da2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:18.080318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b987fc-c30b-4a89-a733-9e1b1fa04da2 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:18.085024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7e797-d6c5-40e0-965c-9601a447b1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:18.085916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7e797-d6c5-40e0-965c-9601a447b1cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:18.086219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7e797-d6c5-40e0-965c-9601a447b1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:18.086491 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7e797-d6c5-40e0-965c-9601a447b1cc admin admin] PATH_INFO: `/v3/users/9932c7fb57394d709a27023f3ae2677e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:18.087693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7e797-d6c5-40e0-965c-9601a447b1cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9932c7fb57394d709a27023f3ae2677e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:18.089610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7e797-d6c5-40e0-965c-9601a447b1cc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:18.296383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e1e7e797-d6c5-40e0-965c-9601a447b1cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9932c7fb57394d709a27023f3ae2677e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:18.309250 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e1e7e797-d6c5-40e0-965c-9601a447b1cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9932c7fb57394d709a27023f3ae2677e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:18.309932 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e1e7e797-d6c5-40e0-965c-9601a447b1cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9932c7fb57394d709a27023f3ae2677e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:18.364773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e1e7e797-d6c5-40e0-965c-9601a447b1cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9932c7fb57394d709a27023f3ae2677e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:18.365375 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e1e7e797-d6c5-40e0-965c-9601a447b1cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9932c7fb57394d709a27023f3ae2677e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:18.378725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1346/2662] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:25:18 2022] DELETE /identity/v3/users/9932c7fb57394d709a27023f3ae2677e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:18.601136 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616d2ef-f6da-4fbf-b531-315c3b37f42f service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:18.613028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616d2ef-f6da-4fbf-b531-315c3b37f42f service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:18.671144 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5e7c0-5bfc-4f65-b5cb-2f58238754a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:18.671761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5e7c0-5bfc-4f65-b5cb-2f58238754a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:18.672035 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5e7c0-5bfc-4f65-b5cb-2f58238754a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:18.672304 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5e7c0-5bfc-4f65-b5cb-2f58238754a1 admin admin] PATH_INFO: `/v3/projects/d6a6c776cba64be9a1710815c0b0f952` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:18.673402 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5e7c0-5bfc-4f65-b5cb-2f58238754a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6a6c776cba64be9a1710815c0b0f952)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:18.675373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5e7c0-5bfc-4f65-b5cb-2f58238754a1 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:18.775413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6ea5e7c0-5bfc-4f65-b5cb-2f58238754a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6a6c776cba64be9a1710815c0b0f952', 'request_id': 'req-6ea5e7c0-5bfc-4f65-b5cb-2f58238754a1'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:18.777266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:18.781821 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6ea5e7c0-5bfc-4f65-b5cb-2f58238754a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6a6c776cba64be9a1710815c0b0f952', 'request_id': 'req-6ea5e7c0-5bfc-4f65-b5cb-2f58238754a1'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:18.877830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6ea5e7c0-5bfc-4f65-b5cb-2f58238754a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6a6c776cba64be9a1710815c0b0f952', 'request_id': 'req-6ea5e7c0-5bfc-4f65-b5cb-2f58238754a1'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:18.889262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6ea5e7c0-5bfc-4f65-b5cb-2f58238754a1 admin admin] The token cache is being invalidate because project d6a6c776cba64be9a1710815c0b0f952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:25:18.890030 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6ea5e7c0-5bfc-4f65-b5cb-2f58238754a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:18.891074 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6ea5e7c0-5bfc-4f65-b5cb-2f58238754a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:18.892601 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1317/2663] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:25:18 2022] DELETE /identity/v3/projects/d6a6c776cba64be9a1710815c0b0f952 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:18.897592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfb276-1d78-4b95-9916-ad7f59672f2a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:18.903603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfb276-1d78-4b95-9916-ad7f59672f2a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:18.903603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfb276-1d78-4b95-9916-ad7f59672f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:18.918885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0cbfb276-1d78-4b95-9916-ad7f59672f2a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:18.920115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0cbfb276-1d78-4b95-9916-ad7f59672f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:18.978908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1347/2664] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:18.986417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1043cc8-d330-4731-be06-99aa4a1ef7cf None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:18.986719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1043cc8-d330-4731-be06-99aa4a1ef7cf None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:18.987019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1043cc8-d330-4731-be06-99aa4a1ef7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:19.006780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-b1043cc8-d330-4731-be06-99aa4a1ef7cf None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:19.008159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b1043cc8-d330-4731-be06-99aa4a1ef7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:19.016497 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1318/2665] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:19.020637 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfb276-1d78-4b95-9916-ad7f59672f2a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:19.030806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfb276-1d78-4b95-9916-ad7f59672f2a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:19.035324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ed68b-dbfe-4d5b-8faf-c2ea8dedd8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:19.035911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ed68b-dbfe-4d5b-8faf-c2ea8dedd8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:19.036184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ed68b-dbfe-4d5b-8faf-c2ea8dedd8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:19.036449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ed68b-dbfe-4d5b-8faf-c2ea8dedd8d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:19.037083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ed68b-dbfe-4d5b-8faf-c2ea8dedd8d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:25:19.037377 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ed68b-dbfe-4d5b-8faf-c2ea8dedd8d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:19.039028 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ed68b-dbfe-4d5b-8faf-c2ea8dedd8d2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:19.051482 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1348/2666] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:25:19.055795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1043cc8-d330-4731-be06-99aa4a1ef7cf None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:19.064951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1043cc8-d330-4731-be06-99aa4a1ef7cf None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:19.069687 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcad1a4-18bc-403c-891c-fda78b1a0638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:19.070278 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcad1a4-18bc-403c-891c-fda78b1a0638 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:19.070587 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcad1a4-18bc-403c-891c-fda78b1a0638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:19.070858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcad1a4-18bc-403c-891c-fda78b1a0638 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:19.071554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcad1a4-18bc-403c-891c-fda78b1a0638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:19.073455 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcad1a4-18bc-403c-891c-fda78b1a0638 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:19.167184 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1319/2667] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:25:19 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:25:19.171210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ed68b-dbfe-4d5b-8faf-c2ea8dedd8d2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:19.181304 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ed68b-dbfe-4d5b-8faf-c2ea8dedd8d2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:19.238201 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ad486-e91e-4bb1-8f51-8fb15d70fac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:19.238826 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ad486-e91e-4bb1-8f51-8fb15d70fac2 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:19.239132 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ad486-e91e-4bb1-8f51-8fb15d70fac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:19.239403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ad486-e91e-4bb1-8f51-8fb15d70fac2 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:19.240045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ad486-e91e-4bb1-8f51-8fb15d70fac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:19.241989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ad486-e91e-4bb1-8f51-8fb15d70fac2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:19.249935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:19.362560 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1349/2668] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:25:19 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:25:19.366612 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcad1a4-18bc-403c-891c-fda78b1a0638 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:19.377642 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcad1a4-18bc-403c-891c-fda78b1a0638 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:19.382573 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddcfdf-8417-4606-868d-6bd444edece5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:19.388880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddcfdf-8417-4606-868d-6bd444edece5 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:19.388880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddcfdf-8417-4606-868d-6bd444edece5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:19.388880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddcfdf-8417-4606-868d-6bd444edece5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:19.388880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcddcfdf-8417-4606-868d-6bd444edece5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:19.388880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcddcfdf-8417-4606-868d-6bd444edece5 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:19.396886 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1320/2669] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:25:19.402096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ad486-e91e-4bb1-8f51-8fb15d70fac2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:19.412263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ad486-e91e-4bb1-8f51-8fb15d70fac2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:19.416904 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e920d58-a0a9-4865-9802-8714e1baba4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:19.417533 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e920d58-a0a9-4865-9802-8714e1baba4b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:19.417807 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e920d58-a0a9-4865-9802-8714e1baba4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:19.418072 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e920d58-a0a9-4865-9802-8714e1baba4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:19.418714 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e920d58-a0a9-4865-9802-8714e1baba4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:19.420830 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e920d58-a0a9-4865-9802-8714e1baba4b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:19.431376 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1350/2670] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:25:19.435439 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddcfdf-8417-4606-868d-6bd444edece5 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:19.445337 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddcfdf-8417-4606-868d-6bd444edece5 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:19.450579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65634c0-3d44-4b32-bb26-c9180e52ace0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:19.451264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65634c0-3d44-4b32-bb26-c9180e52ace0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:19.451547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65634c0-3d44-4b32-bb26-c9180e52ace0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:19.451820 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65634c0-3d44-4b32-bb26-c9180e52ace0 admin admin] PATH_INFO: `/v3/projects/c55402c6b2c74ceb87b34e6ed893f426/users/0c2df04b6e2349d99fa294990e37e023/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:19.466919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65634c0-3d44-4b32-bb26-c9180e52ace0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c55402c6b2c74ceb87b34e6ed893f426, user_id=0c2df04b6e2349d99fa294990e37e023, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:19.469604 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65634c0-3d44-4b32-bb26-c9180e52ace0 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:19.534478 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1321/2671] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:25:19 2022] PUT /identity/v3/projects/c55402c6b2c74ceb87b34e6ed893f426/users/0c2df04b6e2349d99fa294990e37e023/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:19.545631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395e5eb-0581-4f40-b95d-905c5e2562aa None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:19.545916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395e5eb-0581-4f40-b95d-905c5e2562aa None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:19.546187 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395e5eb-0581-4f40-b95d-905c5e2562aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:19.575056 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1395e5eb-0581-4f40-b95d-905c5e2562aa None None] MFA Rules not processed for user `0c2df04b6e2349d99fa294990e37e023`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:19.576275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1395e5eb-0581-4f40-b95d-905c5e2562aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:19.641558 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1351/2672] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:19.652582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65634c0-3d44-4b32-bb26-c9180e52ace0 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:19.654567 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a65634c0-3d44-4b32-bb26-c9180e52ace0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:19.729714 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65634c0-3d44-4b32-bb26-c9180e52ace0 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:19.735503 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3505105-2671-42fc-9cbd-93a3a6357671 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:19.736105 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3505105-2671-42fc-9cbd-93a3a6357671 service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:19.736386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3505105-2671-42fc-9cbd-93a3a6357671 service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:19.736661 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3505105-2671-42fc-9cbd-93a3a6357671 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:19.745451 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3505105-2671-42fc-9cbd-93a3a6357671 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:19.747071 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3505105-2671-42fc-9cbd-93a3a6357671 service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:19.756256 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1322/2673] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:25:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:25:29.637805 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:29.751932 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=66,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:48.298678 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395e5eb-0581-4f40-b95d-905c5e2562aa None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:48.360358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395e5eb-0581-4f40-b95d-905c5e2562aa None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:48.364851 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9969fc-28de-401d-b771-6bcebd98d168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:48.365467 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9969fc-28de-401d-b771-6bcebd98d168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:48.365738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9969fc-28de-401d-b771-6bcebd98d168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:48.366003 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9969fc-28de-401d-b771-6bcebd98d168 admin admin] PATH_INFO: `/v3/users/0c2df04b6e2349d99fa294990e37e023` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:48.367886 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9969fc-28de-401d-b771-6bcebd98d168 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c2df04b6e2349d99fa294990e37e023)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:48.369926 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9969fc-28de-401d-b771-6bcebd98d168 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:48.594326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bd9969fc-28de-401d-b771-6bcebd98d168 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c2df04b6e2349d99fa294990e37e023'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:48.606723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bd9969fc-28de-401d-b771-6bcebd98d168 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c2df04b6e2349d99fa294990e37e023'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:48.607557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bd9969fc-28de-401d-b771-6bcebd98d168 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c2df04b6e2349d99fa294990e37e023'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:48.678477 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bd9969fc-28de-401d-b771-6bcebd98d168 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c2df04b6e2349d99fa294990e37e023'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:48.679821 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bd9969fc-28de-401d-b771-6bcebd98d168 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c2df04b6e2349d99fa294990e37e023'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:48.692813 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1352/2674] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:25:48 2022] DELETE /identity/v3/users/0c2df04b6e2349d99fa294990e37e023 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:48.697954 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be270dc-2ca5-43f3-8945-fe793332a0cc None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:48.698421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be270dc-2ca5-43f3-8945-fe793332a0cc None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:48.698554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be270dc-2ca5-43f3-8945-fe793332a0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:48.718196 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-6be270dc-2ca5-43f3-8945-fe793332a0cc None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:48.719441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6be270dc-2ca5-43f3-8945-fe793332a0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:48.782456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1323/2675] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:48.793054 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9969fc-28de-401d-b771-6bcebd98d168 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:48.794537 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bd9969fc-28de-401d-b771-6bcebd98d168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:48.853857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9969fc-28de-401d-b771-6bcebd98d168 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:48.857416 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5f618-a7cd-48e8-a5ce-1502f309f5df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:48.858023 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5f618-a7cd-48e8-a5ce-1502f309f5df service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:48.858303 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5f618-a7cd-48e8-a5ce-1502f309f5df service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:48.858572 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5f618-a7cd-48e8-a5ce-1502f309f5df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:48.871757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5f618-a7cd-48e8-a5ce-1502f309f5df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:48.873279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5f618-a7cd-48e8-a5ce-1502f309f5df service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:48.883367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1353/2676] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:25:49.216505 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be270dc-2ca5-43f3-8945-fe793332a0cc None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:49.218039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6be270dc-2ca5-43f3-8945-fe793332a0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:49.231064 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be270dc-2ca5-43f3-8945-fe793332a0cc None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:49.235751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618474e6-ea5b-439a-803e-96986d028ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:49.237554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618474e6-ea5b-439a-803e-96986d028ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:49.237554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618474e6-ea5b-439a-803e-96986d028ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:49.237554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618474e6-ea5b-439a-803e-96986d028ee7 admin admin] PATH_INFO: `/v3/users/2b0b24d52238425494bb2192e3615e56` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:49.238039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618474e6-ea5b-439a-803e-96986d028ee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b0b24d52238425494bb2192e3615e56)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:49.240000 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618474e6-ea5b-439a-803e-96986d028ee7 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:49.298327 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5f618-a7cd-48e8-a5ce-1502f309f5df service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:49.299849 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-43a5f618-a7cd-48e8-a5ce-1502f309f5df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:49.314186 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5f618-a7cd-48e8-a5ce-1502f309f5df service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:49.319118 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe1b37-6abd-4a0e-b3df-210a71777533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:49.319835 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe1b37-6abd-4a0e-b3df-210a71777533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:49.320116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe1b37-6abd-4a0e-b3df-210a71777533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:49.320388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe1b37-6abd-4a0e-b3df-210a71777533 admin admin] PATH_INFO: `/v3/projects/c55402c6b2c74ceb87b34e6ed893f426` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:49.321551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe1b37-6abd-4a0e-b3df-210a71777533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c55402c6b2c74ceb87b34e6ed893f426)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:49.323616 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe1b37-6abd-4a0e-b3df-210a71777533 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:49.342150 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:49.471477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-618474e6-ea5b-439a-803e-96986d028ee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b0b24d52238425494bb2192e3615e56'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:49.472068 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-618474e6-ea5b-439a-803e-96986d028ee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b0b24d52238425494bb2192e3615e56'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:49.474640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8cfe1b37-6abd-4a0e-b3df-210a71777533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c55402c6b2c74ceb87b34e6ed893f426', 'request_id': 'req-8cfe1b37-6abd-4a0e-b3df-210a71777533'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:49.481851 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8cfe1b37-6abd-4a0e-b3df-210a71777533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c55402c6b2c74ceb87b34e6ed893f426', 'request_id': 'req-8cfe1b37-6abd-4a0e-b3df-210a71777533'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:49.484838 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-618474e6-ea5b-439a-803e-96986d028ee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b0b24d52238425494bb2192e3615e56'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:49.571021 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8cfe1b37-6abd-4a0e-b3df-210a71777533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c55402c6b2c74ceb87b34e6ed893f426', 'request_id': 'req-8cfe1b37-6abd-4a0e-b3df-210a71777533'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:49.581426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8cfe1b37-6abd-4a0e-b3df-210a71777533 admin admin] The token cache is being invalidate because project c55402c6b2c74ceb87b34e6ed893f426 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:25:49.581707 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8cfe1b37-6abd-4a0e-b3df-210a71777533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:49.582322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8cfe1b37-6abd-4a0e-b3df-210a71777533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:49.583847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1354/2677] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:25:49 2022] DELETE /identity/v3/projects/c55402c6b2c74ceb87b34e6ed893f426 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:25:49.588999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a21bd-19ef-4b56-91ca-e129d17e1999 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:49.589271 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a21bd-19ef-4b56-91ca-e129d17e1999 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:49.589535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a21bd-19ef-4b56-91ca-e129d17e1999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:49.608702 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3a9a21bd-19ef-4b56-91ca-e129d17e1999 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:49.609890 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3a9a21bd-19ef-4b56-91ca-e129d17e1999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:49.642506 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-618474e6-ea5b-439a-803e-96986d028ee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b0b24d52238425494bb2192e3615e56'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:49.646866 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-618474e6-ea5b-439a-803e-96986d028ee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b0b24d52238425494bb2192e3615e56'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:49.657724 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1324/2678] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:25:49 2022] DELETE /identity/v3/users/2b0b24d52238425494bb2192e3615e56 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:49.662466 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7a55a-4700-4f61-b272-b0d976c93760 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:49.662803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7a55a-4700-4f61-b272-b0d976c93760 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:49.663210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7a55a-4700-4f61-b272-b0d976c93760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:49.672233 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1355/2679] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:25:49.680037 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ca06e-72c5-4d88-86ea-e15a92aa2037 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:49.680440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ca06e-72c5-4d88-86ea-e15a92aa2037 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:49.680716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ca06e-72c5-4d88-86ea-e15a92aa2037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:49.683454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-bea7a55a-4700-4f61-b272-b0d976c93760 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:49.684596 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-bea7a55a-4700-4f61-b272-b0d976c93760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:49.701051 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-591ca06e-72c5-4d88-86ea-e15a92aa2037 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:49.702235 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-591ca06e-72c5-4d88-86ea-e15a92aa2037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:49.744969 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1325/2680] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:49.754539 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7a55a-4700-4f61-b272-b0d976c93760 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:49.756079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-bea7a55a-4700-4f61-b272-b0d976c93760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:49.762771 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1356/2681] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:25:49.766910 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ca06e-72c5-4d88-86ea-e15a92aa2037 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:49.771155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:49.777204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ca06e-72c5-4d88-86ea-e15a92aa2037 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:49.782472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6516a-b19e-45f3-ba3c-ee296e8ae934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:49.782472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6516a-b19e-45f3-ba3c-ee296e8ae934 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:49.782750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6516a-b19e-45f3-ba3c-ee296e8ae934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:49.783042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6516a-b19e-45f3-ba3c-ee296e8ae934 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:49.783715 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6516a-b19e-45f3-ba3c-ee296e8ae934 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:25:49.784014 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6516a-b19e-45f3-ba3c-ee296e8ae934 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:49.785613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6516a-b19e-45f3-ba3c-ee296e8ae934 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:49.797152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1357/2682] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 08:25:49.801488 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6516a-b19e-45f3-ba3c-ee296e8ae934 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:49.811414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6516a-b19e-45f3-ba3c-ee296e8ae934 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:49.815493 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7a55a-4700-4f61-b272-b0d976c93760 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:49.816116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972447be-600d-48b6-ba70-11b65362e8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:49.816699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972447be-600d-48b6-ba70-11b65362e8c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:49.816977 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972447be-600d-48b6-ba70-11b65362e8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:49.817249 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972447be-600d-48b6-ba70-11b65362e8c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:49.817873 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972447be-600d-48b6-ba70-11b65362e8c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:49.819204 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a2ed4-f5df-4b6c-b4d1-9d1dd59b439d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:49.820350 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a2ed4-f5df-4b6c-b4d1-9d1dd59b439d service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:49.820480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972447be-600d-48b6-ba70-11b65362e8c5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:49.820628 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a2ed4-f5df-4b6c-b4d1-9d1dd59b439d service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:49.820899 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a2ed4-f5df-4b6c-b4d1-9d1dd59b439d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:49.830041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a2ed4-f5df-4b6c-b4d1-9d1dd59b439d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:49.832817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a2ed4-f5df-4b6c-b4d1-9d1dd59b439d service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:49.843704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1326/2683] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:25:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:25:49.909020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1358/2684] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:25:49 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:25:49.910987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a2ed4-f5df-4b6c-b4d1-9d1dd59b439d service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:49.921796 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a2ed4-f5df-4b6c-b4d1-9d1dd59b439d service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:49.988073 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5cead-2bec-431d-b651-4e15a10951ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:49.988679 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5cead-2bec-431d-b651-4e15a10951ac admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:49.988986 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5cead-2bec-431d-b651-4e15a10951ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:49.989259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5cead-2bec-431d-b651-4e15a10951ac admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:49.989882 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e5cead-2bec-431d-b651-4e15a10951ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:49.992421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e5cead-2bec-431d-b651-4e15a10951ac admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:50.096849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1327/2685] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:25:49 2022] POST /identity/v3/users => generated 284 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:25:50.101204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972447be-600d-48b6-ba70-11b65362e8c5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:50.112905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972447be-600d-48b6-ba70-11b65362e8c5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:50.118535 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62959c5-c360-45e4-9fba-97f967504b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:50.119236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62959c5-c360-45e4-9fba-97f967504b61 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:50.119526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62959c5-c360-45e4-9fba-97f967504b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:50.119776 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62959c5-c360-45e4-9fba-97f967504b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:50.120417 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62959c5-c360-45e4-9fba-97f967504b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:50.122000 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62959c5-c360-45e4-9fba-97f967504b61 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:50.127872 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:50.133427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1359/2686] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:25:50.138035 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5cead-2bec-431d-b651-4e15a10951ac admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:50.153791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5cead-2bec-431d-b651-4e15a10951ac admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:50.159579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f906689-9036-4f69-9d46-ac59f98d0306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:50.165674 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f906689-9036-4f69-9d46-ac59f98d0306 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:50.165674 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f906689-9036-4f69-9d46-ac59f98d0306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:50.165674 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f906689-9036-4f69-9d46-ac59f98d0306 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:50.165674 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f906689-9036-4f69-9d46-ac59f98d0306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:50.165674 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f906689-9036-4f69-9d46-ac59f98d0306 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:50.174472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1328/2687] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:25:50.179549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62959c5-c360-45e4-9fba-97f967504b61 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:50.191790 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62959c5-c360-45e4-9fba-97f967504b61 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:50.197094 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da9d1d-9822-436b-90b6-4f3b68ce76f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:50.197738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da9d1d-9822-436b-90b6-4f3b68ce76f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:50.198494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da9d1d-9822-436b-90b6-4f3b68ce76f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:50.198766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da9d1d-9822-436b-90b6-4f3b68ce76f7 admin admin] PATH_INFO: `/v3/projects/9fb06f544c6944a5beed2737c374fa15/users/1915b54b0ae44aaca0828457e8ecab7a/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:50.218387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da9d1d-9822-436b-90b6-4f3b68ce76f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb06f544c6944a5beed2737c374fa15, user_id=1915b54b0ae44aaca0828457e8ecab7a, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:50.221124 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da9d1d-9822-436b-90b6-4f3b68ce76f7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:50.296822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1360/2688] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:25:50 2022] PUT /identity/v3/projects/9fb06f544c6944a5beed2737c374fa15/users/1915b54b0ae44aaca0828457e8ecab7a/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:50.306162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f906689-9036-4f69-9d46-ac59f98d0306 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:50.307942 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1f906689-9036-4f69-9d46-ac59f98d0306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:50.315354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38a6dc-8c1a-4da8-82ad-1862832e5bfc None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:50.315354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38a6dc-8c1a-4da8-82ad-1862832e5bfc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:50.315354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38a6dc-8c1a-4da8-82ad-1862832e5bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:50.322034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f906689-9036-4f69-9d46-ac59f98d0306 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:50.352444 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fd38a6dc-8c1a-4da8-82ad-1862832e5bfc None None] MFA Rules not processed for user `1915b54b0ae44aaca0828457e8ecab7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:50.353685 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fd38a6dc-8c1a-4da8-82ad-1862832e5bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:50.392295 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febde20-552c-4749-a7fd-da68dc4170f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:50.393266 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febde20-552c-4749-a7fd-da68dc4170f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:50.393548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febde20-552c-4749-a7fd-da68dc4170f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:50.393823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febde20-552c-4749-a7fd-da68dc4170f5 admin admin] PATH_INFO: `/v3/projects/c8115e49b70445419388b5746ea2e296` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:50.395112 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febde20-552c-4749-a7fd-da68dc4170f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8115e49b70445419388b5746ea2e296)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:50.396987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febde20-552c-4749-a7fd-da68dc4170f5 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:50.438185 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1361/2689] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:25:50.439911 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38a6dc-8c1a-4da8-82ad-1862832e5bfc None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:50.451631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38a6dc-8c1a-4da8-82ad-1862832e5bfc None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:50.456602 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e9522-7051-408c-a7de-1f6eab9e4a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:50.457199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e9522-7051-408c-a7de-1f6eab9e4a03 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:50.457582 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e9522-7051-408c-a7de-1f6eab9e4a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:50.466355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e9522-7051-408c-a7de-1f6eab9e4a03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:50.466355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e9522-7051-408c-a7de-1f6eab9e4a03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:50.466355 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e9522-7051-408c-a7de-1f6eab9e4a03 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:50.549782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1362/2690] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:25:50 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:25:50.553752 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e9522-7051-408c-a7de-1f6eab9e4a03 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:50.553929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:50.572097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6febde20-552c-4749-a7fd-da68dc4170f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8115e49b70445419388b5746ea2e296', 'request_id': 'req-6febde20-552c-4749-a7fd-da68dc4170f5'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:50.578735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6febde20-552c-4749-a7fd-da68dc4170f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8115e49b70445419388b5746ea2e296', 'request_id': 'req-6febde20-552c-4749-a7fd-da68dc4170f5'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:50.622472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e9522-7051-408c-a7de-1f6eab9e4a03 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:50.628219 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b84399-0fd3-426d-a728-37ae2b533440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:50.644329 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b84399-0fd3-426d-a728-37ae2b533440 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:50.644329 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b84399-0fd3-426d-a728-37ae2b533440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:50.644329 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b84399-0fd3-426d-a728-37ae2b533440 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:50.644329 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b84399-0fd3-426d-a728-37ae2b533440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:50.644329 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b84399-0fd3-426d-a728-37ae2b533440 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:50.665729 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6febde20-552c-4749-a7fd-da68dc4170f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8115e49b70445419388b5746ea2e296', 'request_id': 'req-6febde20-552c-4749-a7fd-da68dc4170f5'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:50.678087 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6febde20-552c-4749-a7fd-da68dc4170f5 admin admin] The token cache is being invalidate because project c8115e49b70445419388b5746ea2e296 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:25:50.678447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6febde20-552c-4749-a7fd-da68dc4170f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:50.679888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6febde20-552c-4749-a7fd-da68dc4170f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:50.681423 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1329/2691] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:25:50 2022] DELETE /identity/v3/projects/c8115e49b70445419388b5746ea2e296 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:50.686249 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620715e-58fb-4742-8f7f-cfd4d9eec939 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:50.686531 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620715e-58fb-4742-8f7f-cfd4d9eec939 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:50.686827 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620715e-58fb-4742-8f7f-cfd4d9eec939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:50.708323 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-7620715e-58fb-4742-8f7f-cfd4d9eec939 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:50.709538 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7620715e-58fb-4742-8f7f-cfd4d9eec939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:50.717807 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1330/2692] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:25:50.727054 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686401c-3783-4330-88e5-d72e1bf329c8 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:50.727335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686401c-3783-4330-88e5-d72e1bf329c8 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:50.727607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686401c-3783-4330-88e5-d72e1bf329c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:50.735423 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1363/2693] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:25:50 2022] POST /identity/v3/users => generated 291 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:25:50.739476 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b84399-0fd3-426d-a728-37ae2b533440 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:50.741643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f2b84399-0fd3-426d-a728-37ae2b533440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:50.748835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-2686401c-3783-4330-88e5-d72e1bf329c8 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:50.750013 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-2686401c-3783-4330-88e5-d72e1bf329c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:50.754464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b84399-0fd3-426d-a728-37ae2b533440 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:50.759440 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1331/2694] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:25:50.759778 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda844a5-8242-487d-a770-245cf6165f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:50.760366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda844a5-8242-487d-a770-245cf6165f90 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:50.760641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda844a5-8242-487d-a770-245cf6165f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:50.760905 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda844a5-8242-487d-a770-245cf6165f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:50.761532 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda844a5-8242-487d-a770-245cf6165f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:50.763142 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda844a5-8242-487d-a770-245cf6165f90 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:50.763614 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686401c-3783-4330-88e5-d72e1bf329c8 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:50.772927 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686401c-3783-4330-88e5-d72e1bf329c8 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:50.774038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1364/2695] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:25:50.778077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda844a5-8242-487d-a770-245cf6165f90 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:50.780548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f3906-88ac-4ba1-8824-eb494d87071d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:50.781156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f3906-88ac-4ba1-8824-eb494d87071d admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:50.781434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f3906-88ac-4ba1-8824-eb494d87071d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:50.781706 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f3906-88ac-4ba1-8824-eb494d87071d admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:50.782353 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f3906-88ac-4ba1-8824-eb494d87071d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:25:50.782656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f3906-88ac-4ba1-8824-eb494d87071d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:50.784575 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f3906-88ac-4ba1-8824-eb494d87071d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:50.788435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda844a5-8242-487d-a770-245cf6165f90 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:50.793147 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c359bb-4ce3-40ed-b7bf-b5d0a92259c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:50.793780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c359bb-4ce3-40ed-b7bf-b5d0a92259c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:50.794961 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c359bb-4ce3-40ed-b7bf-b5d0a92259c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:50.794961 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c359bb-4ce3-40ed-b7bf-b5d0a92259c7 admin admin] PATH_INFO: `/v3/projects/a4d690f0c2b84f12918387ef96321802/users/e4a3899ced084a259b314671a20522f9/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:50.800563 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1332/2696] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 08:25:50.801534 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f3906-88ac-4ba1-8824-eb494d87071d admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:50.810592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c359bb-4ce3-40ed-b7bf-b5d0a92259c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4d690f0c2b84f12918387ef96321802, user_id=e4a3899ced084a259b314671a20522f9, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:50.811712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f3906-88ac-4ba1-8824-eb494d87071d admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:50.813593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c359bb-4ce3-40ed-b7bf-b5d0a92259c7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:50.816342 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6cc22-b292-49ed-b946-c430f70c963a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:50.816923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6cc22-b292-49ed-b946-c430f70c963a admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:50.817195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6cc22-b292-49ed-b946-c430f70c963a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:50.817469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6cc22-b292-49ed-b946-c430f70c963a admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:50.818082 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6cc22-b292-49ed-b946-c430f70c963a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:50.820748 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6cc22-b292-49ed-b946-c430f70c963a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:50.900252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:50.905852 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1365/2697] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:25:50 2022] PUT /identity/v3/projects/a4d690f0c2b84f12918387ef96321802/users/e4a3899ced084a259b314671a20522f9/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:25:50.910023 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c359bb-4ce3-40ed-b7bf-b5d0a92259c7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:50.921837 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1333/2698] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:25:50 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:25:50.922836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6cc22-b292-49ed-b946-c430f70c963a admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:50.992966 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c359bb-4ce3-40ed-b7bf-b5d0a92259c7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:50.997235 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6cc22-b292-49ed-b946-c430f70c963a admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:51.002264 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a560f06-dae9-473d-b4b7-cdf6bf083236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:51.002264 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a560f06-dae9-473d-b4b7-cdf6bf083236 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:51.002264 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a560f06-dae9-473d-b4b7-cdf6bf083236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:51.002264 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a560f06-dae9-473d-b4b7-cdf6bf083236 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:51.002669 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a560f06-dae9-473d-b4b7-cdf6bf083236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:51.002669 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a560f06-dae9-473d-b4b7-cdf6bf083236 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:51.004504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6cc17-526e-4630-9398-3340e73c60ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:51.005111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6cc17-526e-4630-9398-3340e73c60ee admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:51.005389 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6cc17-526e-4630-9398-3340e73c60ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:51.005663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6cc17-526e-4630-9398-3340e73c60ee admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:51.006294 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6cc17-526e-4630-9398-3340e73c60ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:51.008911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6cc17-526e-4630-9398-3340e73c60ee admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:51.012514 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1366/2699] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:25:51.016872 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a560f06-dae9-473d-b4b7-cdf6bf083236 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:51.027097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a560f06-dae9-473d-b4b7-cdf6bf083236 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:51.031920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0549d8c-4276-4c59-b1cf-123cd73e9f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:51.035080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0549d8c-4276-4c59-b1cf-123cd73e9f8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:51.035080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0549d8c-4276-4c59-b1cf-123cd73e9f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:51.035080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0549d8c-4276-4c59-b1cf-123cd73e9f8a admin admin] PATH_INFO: `/v3/projects/a4d690f0c2b84f12918387ef96321802/users/e4a3899ced084a259b314671a20522f9/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:51.035080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0549d8c-4276-4c59-b1cf-123cd73e9f8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4d690f0c2b84f12918387ef96321802, user_id=e4a3899ced084a259b314671a20522f9, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:51.037757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0549d8c-4276-4c59-b1cf-123cd73e9f8a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:51.048370 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1367/2700] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:25:51 2022] PUT /identity/v3/projects/a4d690f0c2b84f12918387ef96321802/users/e4a3899ced084a259b314671a20522f9/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:25:51.063803 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6d5a7-ae25-4746-998e-8a23930549ff None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:51.063803 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6d5a7-ae25-4746-998e-8a23930549ff None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:51.063803 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6d5a7-ae25-4746-998e-8a23930549ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:51.088339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-03c6d5a7-ae25-4746-998e-8a23930549ff None None] MFA Rules not processed for user `e4a3899ced084a259b314671a20522f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:51.089606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-03c6d5a7-ae25-4746-998e-8a23930549ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:51.111413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1334/2701] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:25:50 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:25:51.115293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6cc17-526e-4630-9398-3340e73c60ee admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:51.127484 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6cc17-526e-4630-9398-3340e73c60ee admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:51.166998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1368/2702] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:25:51.178604 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6d5a7-ae25-4746-998e-8a23930549ff None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:51.181230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-03c6d5a7-ae25-4746-998e-8a23930549ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:51.196719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70128c21-d670-418e-80f7-ed0c39a12637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:51.197450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70128c21-d670-418e-80f7-ed0c39a12637 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:51.197732 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70128c21-d670-418e-80f7-ed0c39a12637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:51.198038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70128c21-d670-418e-80f7-ed0c39a12637 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:51.198735 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70128c21-d670-418e-80f7-ed0c39a12637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:51.200536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70128c21-d670-418e-80f7-ed0c39a12637 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:51.210980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1335/2703] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:25:51.215012 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70128c21-d670-418e-80f7-ed0c39a12637 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:51.220318 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:51.225289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70128c21-d670-418e-80f7-ed0c39a12637 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:51.230670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e31c5-0fc7-49a1-9546-b4fa1520196b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:51.231298 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e31c5-0fc7-49a1-9546-b4fa1520196b admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:51.231605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e31c5-0fc7-49a1-9546-b4fa1520196b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:51.231875 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e31c5-0fc7-49a1-9546-b4fa1520196b admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:51.232508 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e31c5-0fc7-49a1-9546-b4fa1520196b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:51.234104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e31c5-0fc7-49a1-9546-b4fa1520196b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:51.238006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6d5a7-ae25-4746-998e-8a23930549ff None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:51.242647 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fc965-b7a9-4eef-83eb-7140d460fc25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:51.243279 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fc965-b7a9-4eef-83eb-7140d460fc25 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:51.243557 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fc965-b7a9-4eef-83eb-7140d460fc25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:51.243828 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fc965-b7a9-4eef-83eb-7140d460fc25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:51.245268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1336/2704] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:25:51.249630 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e31c5-0fc7-49a1-9546-b4fa1520196b admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:51.255391 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fc965-b7a9-4eef-83eb-7140d460fc25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:51.257642 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fc965-b7a9-4eef-83eb-7140d460fc25 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:51.259940 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e31c5-0fc7-49a1-9546-b4fa1520196b admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:51.267914 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6646d-cf06-4d63-a976-a5966de3f85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:51.268273 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1369/2705] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:25:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:25:51.270355 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6646d-cf06-4d63-a976-a5966de3f85a admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:51.270355 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6646d-cf06-4d63-a976-a5966de3f85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:51.270355 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6646d-cf06-4d63-a976-a5966de3f85a admin admin] PATH_INFO: `/v3/projects/fd1ef1a189d5485fb810e17c945b295f/users/79d29aa9482f4c1f9b1554333d85d80a/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:51.285515 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6646d-cf06-4d63-a976-a5966de3f85a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd1ef1a189d5485fb810e17c945b295f, user_id=79d29aa9482f4c1f9b1554333d85d80a, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:51.289066 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6646d-cf06-4d63-a976-a5966de3f85a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:51.387545 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1337/2706] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:25:51 2022] PUT /identity/v3/projects/fd1ef1a189d5485fb810e17c945b295f/users/79d29aa9482f4c1f9b1554333d85d80a/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:25:51.393063 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54695083-cfdf-4a32-9209-d8fc470eeda5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:51.393560 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54695083-cfdf-4a32-9209-d8fc470eeda5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:51.393669 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54695083-cfdf-4a32-9209-d8fc470eeda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:51.412925 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-54695083-cfdf-4a32-9209-d8fc470eeda5 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:51.414878 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-54695083-cfdf-4a32-9209-d8fc470eeda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:51.419181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:51.478646 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1370/2707] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:51.488454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6646d-cf06-4d63-a976-a5966de3f85a admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:51.490324 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-ebc6646d-cf06-4d63-a976-a5966de3f85a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:51.549850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6646d-cf06-4d63-a976-a5966de3f85a admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:51.553835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5fe0e-922b-422d-9c55-42414b297cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:51.554472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5fe0e-922b-422d-9c55-42414b297cda service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:51.555352 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5fe0e-922b-422d-9c55-42414b297cda service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:51.555650 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5fe0e-922b-422d-9c55-42414b297cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:51.566470 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5fe0e-922b-422d-9c55-42414b297cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:51.566775 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5fe0e-922b-422d-9c55-42414b297cda service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:51.579191 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1338/2708] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:25:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:25:53.199120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54695083-cfdf-4a32-9209-d8fc470eeda5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:53.201358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-54695083-cfdf-4a32-9209-d8fc470eeda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:53.213763 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54695083-cfdf-4a32-9209-d8fc470eeda5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:53.218254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7e848-d854-46e5-8557-8827fc25c0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:53.218864 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7e848-d854-46e5-8557-8827fc25c0eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:53.219195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7e848-d854-46e5-8557-8827fc25c0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:53.219462 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7e848-d854-46e5-8557-8827fc25c0eb admin admin] PATH_INFO: `/v3/users/7f32659d61b64b43942ac966674d34e6` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:53.220549 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7e848-d854-46e5-8557-8827fc25c0eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f32659d61b64b43942ac966674d34e6)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:53.222461 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7e848-d854-46e5-8557-8827fc25c0eb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:53.440372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c1d7e848-d854-46e5-8557-8827fc25c0eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f32659d61b64b43942ac966674d34e6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:53.461284 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c1d7e848-d854-46e5-8557-8827fc25c0eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f32659d61b64b43942ac966674d34e6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:53.462208 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c1d7e848-d854-46e5-8557-8827fc25c0eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f32659d61b64b43942ac966674d34e6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:53.517582 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c1d7e848-d854-46e5-8557-8827fc25c0eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f32659d61b64b43942ac966674d34e6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:53.522285 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c1d7e848-d854-46e5-8557-8827fc25c0eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f32659d61b64b43942ac966674d34e6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:53.534655 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1371/2709] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:25:53 2022] DELETE /identity/v3/users/7f32659d61b64b43942ac966674d34e6 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:53.539842 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c60b74-76ef-4ba7-85a1-4920570e7268 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:53.540459 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c60b74-76ef-4ba7-85a1-4920570e7268 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:53.540459 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c60b74-76ef-4ba7-85a1-4920570e7268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:53.559806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-47c60b74-76ef-4ba7-85a1-4920570e7268 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:53.561258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-47c60b74-76ef-4ba7-85a1-4920570e7268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:53.624602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1339/2710] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:53.635506 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7e848-d854-46e5-8557-8827fc25c0eb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:53.636997 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c1d7e848-d854-46e5-8557-8827fc25c0eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:53.727178 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7e848-d854-46e5-8557-8827fc25c0eb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:53.730794 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7a0a1-0905-4d9a-a062-52fda2227058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:53.731433 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7a0a1-0905-4d9a-a062-52fda2227058 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:53.734172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7a0a1-0905-4d9a-a062-52fda2227058 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:53.734172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7a0a1-0905-4d9a-a062-52fda2227058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:53.741238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7a0a1-0905-4d9a-a062-52fda2227058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:53.743292 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7a0a1-0905-4d9a-a062-52fda2227058 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:53.755124 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1372/2711] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:25:54.233557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c60b74-76ef-4ba7-85a1-4920570e7268 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:54.235245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-47c60b74-76ef-4ba7-85a1-4920570e7268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:54.256731 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c60b74-76ef-4ba7-85a1-4920570e7268 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:54.262063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ff82a-49cb-4b32-b397-75d2d040ea12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:54.262712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ff82a-49cb-4b32-b397-75d2d040ea12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:54.263033 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ff82a-49cb-4b32-b397-75d2d040ea12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:54.263302 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ff82a-49cb-4b32-b397-75d2d040ea12 admin admin] PATH_INFO: `/v3/projects/0285a43821d047be94b5265be8123977` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:54.264540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ff82a-49cb-4b32-b397-75d2d040ea12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0285a43821d047be94b5265be8123977)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:54.266671 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ff82a-49cb-4b32-b397-75d2d040ea12 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:54.429000 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fa3ff82a-49cb-4b32-b397-75d2d040ea12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0285a43821d047be94b5265be8123977', 'request_id': 'req-fa3ff82a-49cb-4b32-b397-75d2d040ea12'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:54.435417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fa3ff82a-49cb-4b32-b397-75d2d040ea12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0285a43821d047be94b5265be8123977', 'request_id': 'req-fa3ff82a-49cb-4b32-b397-75d2d040ea12'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:54.529587 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fa3ff82a-49cb-4b32-b397-75d2d040ea12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0285a43821d047be94b5265be8123977', 'request_id': 'req-fa3ff82a-49cb-4b32-b397-75d2d040ea12'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:54.537319 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:54.548392 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fa3ff82a-49cb-4b32-b397-75d2d040ea12 admin admin] The token cache is being invalidate because project 0285a43821d047be94b5265be8123977 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:25:54.548955 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fa3ff82a-49cb-4b32-b397-75d2d040ea12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:54.549424 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-fa3ff82a-49cb-4b32-b397-75d2d040ea12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:54.551401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1340/2712] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:25:54 2022] DELETE /identity/v3/projects/0285a43821d047be94b5265be8123977 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:54.555670 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7a0a1-0905-4d9a-a062-52fda2227058 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:54.567691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7a0a1-0905-4d9a-a062-52fda2227058 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:54.591583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:54.633336 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768699d-18db-402d-8a26-82a3b493eb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:54.633973 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768699d-18db-402d-8a26-82a3b493eb11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:54.634247 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768699d-18db-402d-8a26-82a3b493eb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:54.634513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768699d-18db-402d-8a26-82a3b493eb11 admin admin] PATH_INFO: `/v3/users/39b8d7d194ac4f2b96eb359456890fef` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:54.635680 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768699d-18db-402d-8a26-82a3b493eb11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39b8d7d194ac4f2b96eb359456890fef)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:54.637709 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768699d-18db-402d-8a26-82a3b493eb11 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:54.644233 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ff82a-49cb-4b32-b397-75d2d040ea12 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:54.645926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-fa3ff82a-49cb-4b32-b397-75d2d040ea12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:54.666420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ff82a-49cb-4b32-b397-75d2d040ea12 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:54.673052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0a64f-3711-43ae-9fb6-ed3ba424c423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:54.673764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0a64f-3711-43ae-9fb6-ed3ba424c423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:54.674248 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0a64f-3711-43ae-9fb6-ed3ba424c423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:54.674577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0a64f-3711-43ae-9fb6-ed3ba424c423 admin admin] PATH_INFO: `/v3/users/1915b54b0ae44aaca0828457e8ecab7a` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:54.675830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0a64f-3711-43ae-9fb6-ed3ba424c423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1915b54b0ae44aaca0828457e8ecab7a)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:54.677993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0a64f-3711-43ae-9fb6-ed3ba424c423 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:54.810604 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2768699d-18db-402d-8a26-82a3b493eb11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39b8d7d194ac4f2b96eb359456890fef'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:54.823161 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2768699d-18db-402d-8a26-82a3b493eb11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39b8d7d194ac4f2b96eb359456890fef'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:54.823980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2768699d-18db-402d-8a26-82a3b493eb11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39b8d7d194ac4f2b96eb359456890fef'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:54.895866 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bea0a64f-3711-43ae-9fb6-ed3ba424c423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1915b54b0ae44aaca0828457e8ecab7a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:54.896891 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bea0a64f-3711-43ae-9fb6-ed3ba424c423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1915b54b0ae44aaca0828457e8ecab7a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:54.905533 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2768699d-18db-402d-8a26-82a3b493eb11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39b8d7d194ac4f2b96eb359456890fef'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:54.906520 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2768699d-18db-402d-8a26-82a3b493eb11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39b8d7d194ac4f2b96eb359456890fef'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:54.912798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bea0a64f-3711-43ae-9fb6-ed3ba424c423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1915b54b0ae44aaca0828457e8ecab7a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:54.923630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1373/2713] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:25:54 2022] DELETE /identity/v3/users/39b8d7d194ac4f2b96eb359456890fef => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:54.997913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bea0a64f-3711-43ae-9fb6-ed3ba424c423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1915b54b0ae44aaca0828457e8ecab7a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:55.000025 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bea0a64f-3711-43ae-9fb6-ed3ba424c423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1915b54b0ae44aaca0828457e8ecab7a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:55.028544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1341/2714] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:25:54 2022] DELETE /identity/v3/users/1915b54b0ae44aaca0828457e8ecab7a => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:25:55.033011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b86201-d95f-4b20-a2f8-08b905e6d6bf None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:55.033502 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b86201-d95f-4b20-a2f8-08b905e6d6bf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:55.034730 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b86201-d95f-4b20-a2f8-08b905e6d6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:55.064641 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-44b86201-d95f-4b20-a2f8-08b905e6d6bf None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:55.065830 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-44b86201-d95f-4b20-a2f8-08b905e6d6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:55.141633 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1374/2715] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:25:55.156484 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0a64f-3711-43ae-9fb6-ed3ba424c423 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:55.157944 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-bea0a64f-3711-43ae-9fb6-ed3ba424c423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:55.235802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0a64f-3711-43ae-9fb6-ed3ba424c423 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:55.239939 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710f618-2ed2-45fb-90ee-15c77899a14c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:55.240593 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710f618-2ed2-45fb-90ee-15c77899a14c service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:55.240868 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710f618-2ed2-45fb-90ee-15c77899a14c service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:55.241132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710f618-2ed2-45fb-90ee-15c77899a14c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:55.251179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710f618-2ed2-45fb-90ee-15c77899a14c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:55.252627 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710f618-2ed2-45fb-90ee-15c77899a14c service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:55.268091 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1342/2716] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:25:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:25:55.417461 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b86201-d95f-4b20-a2f8-08b905e6d6bf None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:55.419762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-44b86201-d95f-4b20-a2f8-08b905e6d6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:55.444272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b86201-d95f-4b20-a2f8-08b905e6d6bf None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:55.450441 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7334f3-0234-443e-ba22-ccf6d2ef0892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:55.452320 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7334f3-0234-443e-ba22-ccf6d2ef0892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:55.452720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7334f3-0234-443e-ba22-ccf6d2ef0892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:55.453086 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7334f3-0234-443e-ba22-ccf6d2ef0892 admin admin] PATH_INFO: `/v3/projects/3bdfd96f67454aed909b7b8d317c89b9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:55.455900 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7334f3-0234-443e-ba22-ccf6d2ef0892 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bdfd96f67454aed909b7b8d317c89b9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:55.458801 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7334f3-0234-443e-ba22-ccf6d2ef0892 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:55.612631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd7334f3-0234-443e-ba22-ccf6d2ef0892 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bdfd96f67454aed909b7b8d317c89b9', 'request_id': 'req-dd7334f3-0234-443e-ba22-ccf6d2ef0892'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:55.623866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd7334f3-0234-443e-ba22-ccf6d2ef0892 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bdfd96f67454aed909b7b8d317c89b9', 'request_id': 'req-dd7334f3-0234-443e-ba22-ccf6d2ef0892'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:55.699036 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd7334f3-0234-443e-ba22-ccf6d2ef0892 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bdfd96f67454aed909b7b8d317c89b9', 'request_id': 'req-dd7334f3-0234-443e-ba22-ccf6d2ef0892'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:55.716982 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd7334f3-0234-443e-ba22-ccf6d2ef0892 admin admin] The token cache is being invalidate because project 3bdfd96f67454aed909b7b8d317c89b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:25:55.717797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd7334f3-0234-443e-ba22-ccf6d2ef0892 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:55.720027 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dd7334f3-0234-443e-ba22-ccf6d2ef0892 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:55.732915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1375/2717] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:25:55 2022] DELETE /identity/v3/projects/3bdfd96f67454aed909b7b8d317c89b9 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:55.737740 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf0ca4-a765-4973-a1da-74ba69e472dd None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:55.738841 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf0ca4-a765-4973-a1da-74ba69e472dd None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:55.739148 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf0ca4-a765-4973-a1da-74ba69e472dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:55.783923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-71bf0ca4-a765-4973-a1da-74ba69e472dd None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:55.785155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-71bf0ca4-a765-4973-a1da-74ba69e472dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:55.866156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:55.916280 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1343/2718] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:55.924685 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bac29-9488-42ba-a978-59feddbb5e9b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:55.924970 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bac29-9488-42ba-a978-59feddbb5e9b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:55.925314 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bac29-9488-42ba-a978-59feddbb5e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:55.953493 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ce2bac29-9488-42ba-a978-59feddbb5e9b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:55.955135 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ce2bac29-9488-42ba-a978-59feddbb5e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:55.964672 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1376/2719] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:55.968873 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf0ca4-a765-4973-a1da-74ba69e472dd None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:55.980228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf0ca4-a765-4973-a1da-74ba69e472dd None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:55.985575 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9fff0-25cb-4fed-b6c7-2b2464850699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:55.986579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9fff0-25cb-4fed-b6c7-2b2464850699 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:55.987067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9fff0-25cb-4fed-b6c7-2b2464850699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:55.987505 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9fff0-25cb-4fed-b6c7-2b2464850699 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:55.988391 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9fff0-25cb-4fed-b6c7-2b2464850699 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:25:55.988921 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9fff0-25cb-4fed-b6c7-2b2464850699 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:55.990750 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9fff0-25cb-4fed-b6c7-2b2464850699 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:55.995442 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:56.013602 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1344/2720] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:25:56.014721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bac29-9488-42ba-a978-59feddbb5e9b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:56.025764 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9fff0-25cb-4fed-b6c7-2b2464850699 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:56.028000 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7bf9fff0-25cb-4fed-b6c7-2b2464850699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:56.030258 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bac29-9488-42ba-a978-59feddbb5e9b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:56.035339 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9ebc5-2696-416e-8f92-5758493041a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:56.035909 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9ebc5-2696-416e-8f92-5758493041a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:56.036184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9ebc5-2696-416e-8f92-5758493041a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:56.036627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9ebc5-2696-416e-8f92-5758493041a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:56.037462 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d9ebc5-2696-416e-8f92-5758493041a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:56.041742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d9ebc5-2696-416e-8f92-5758493041a9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:56.068284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9fff0-25cb-4fed-b6c7-2b2464850699 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:56.075168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:56.076036 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:56.076313 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:56.076585 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e admin admin] PATH_INFO: `/v3/projects/9fb06f544c6944a5beed2737c374fa15` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:56.079016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fb06f544c6944a5beed2737c374fa15)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:56.080031 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:56.125774 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1377/2721] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:25:56 2022] POST /identity/v3/projects => generated 393 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:25:56.130029 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9ebc5-2696-416e-8f92-5758493041a9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:56.194268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fb06f544c6944a5beed2737c374fa15', 'request_id': 'req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:56.202377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fb06f544c6944a5beed2737c374fa15', 'request_id': 'req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:56.229013 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9ebc5-2696-416e-8f92-5758493041a9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:56.279718 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fb06f544c6944a5beed2737c374fa15', 'request_id': 'req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:56.296577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e admin admin] The token cache is being invalidate because project 9fb06f544c6944a5beed2737c374fa15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:25:56.296877 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:56.299669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:56.302321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1345/2722] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:25:56 2022] DELETE /identity/v3/projects/9fb06f544c6944a5beed2737c374fa15 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:25:56.305350 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:56.308193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:56.312501 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fb011-33c0-4dc9-b84a-c79a597b6b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:56.313129 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fb011-33c0-4dc9-b84a-c79a597b6b15 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:56.313462 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fb011-33c0-4dc9-b84a-c79a597b6b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:56.313738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fb011-33c0-4dc9-b84a-c79a597b6b15 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:56.314436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fb011-33c0-4dc9-b84a-c79a597b6b15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:56.316825 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fb011-33c0-4dc9-b84a-c79a597b6b15 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:56.333872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3a6fb-062a-4ea8-a9f9-fc7a695e4a6e admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:56.345898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877c3a8-8a19-4339-85a7-5a81bb7b8c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:56.345898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877c3a8-8a19-4339-85a7-5a81bb7b8c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:56.345898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877c3a8-8a19-4339-85a7-5a81bb7b8c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:56.345898 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877c3a8-8a19-4339-85a7-5a81bb7b8c2a admin admin] PATH_INFO: `/v3/users/e4a3899ced084a259b314671a20522f9` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:56.346654 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877c3a8-8a19-4339-85a7-5a81bb7b8c2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4a3899ced084a259b314671a20522f9)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:56.348837 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877c3a8-8a19-4339-85a7-5a81bb7b8c2a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:56.449074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1378/2723] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:25:56 2022] POST /identity/v3/users => generated 297 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:25:56.452895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fb011-33c0-4dc9-b84a-c79a597b6b15 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:56.454591 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d34fb011-33c0-4dc9-b84a-c79a597b6b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:56.474514 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fb011-33c0-4dc9-b84a-c79a597b6b15 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:56.480199 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195e3f4-b662-47f1-bffd-1cd998206175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:56.480805 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195e3f4-b662-47f1-bffd-1cd998206175 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:56.481082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195e3f4-b662-47f1-bffd-1cd998206175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:56.481352 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195e3f4-b662-47f1-bffd-1cd998206175 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:56.481996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195e3f4-b662-47f1-bffd-1cd998206175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:56.483691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195e3f4-b662-47f1-bffd-1cd998206175 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:56.495204 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1379/2724] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:25:56.499212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195e3f4-b662-47f1-bffd-1cd998206175 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:56.509798 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195e3f4-b662-47f1-bffd-1cd998206175 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:56.516344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770ee4e-f66a-4fc3-a2b3-73e44f440d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:56.517106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770ee4e-f66a-4fc3-a2b3-73e44f440d4c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:56.517507 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770ee4e-f66a-4fc3-a2b3-73e44f440d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:56.517929 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770ee4e-f66a-4fc3-a2b3-73e44f440d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:56.518792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3770ee4e-f66a-4fc3-a2b3-73e44f440d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:56.520571 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3770ee4e-f66a-4fc3-a2b3-73e44f440d4c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:56.536413 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1380/2725] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:25:56.537002 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770ee4e-f66a-4fc3-a2b3-73e44f440d4c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:56.548601 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770ee4e-f66a-4fc3-a2b3-73e44f440d4c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:56.558459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86359ef-7c46-4589-8bdb-6cda26a3a5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:56.562221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86359ef-7c46-4589-8bdb-6cda26a3a5b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:56.562221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86359ef-7c46-4589-8bdb-6cda26a3a5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:56.562221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86359ef-7c46-4589-8bdb-6cda26a3a5b8 admin admin] PATH_INFO: `/v3/projects/71cd1bd1ecb846ddb8e14f6fe7dd9d64/users/bdb4c94cb50246e59a8c34708a4382df/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:56.575750 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a877c3a8-8a19-4339-85a7-5a81bb7b8c2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4a3899ced084a259b314671a20522f9'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:56.576547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a877c3a8-8a19-4339-85a7-5a81bb7b8c2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4a3899ced084a259b314671a20522f9'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:56.577313 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86359ef-7c46-4589-8bdb-6cda26a3a5b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71cd1bd1ecb846ddb8e14f6fe7dd9d64, user_id=bdb4c94cb50246e59a8c34708a4382df, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:56.580113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86359ef-7c46-4589-8bdb-6cda26a3a5b8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:56.591354 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a877c3a8-8a19-4339-85a7-5a81bb7b8c2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4a3899ced084a259b314671a20522f9'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:56.656195 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a877c3a8-8a19-4339-85a7-5a81bb7b8c2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4a3899ced084a259b314671a20522f9'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:56.656887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:56.658200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a877c3a8-8a19-4339-85a7-5a81bb7b8c2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4a3899ced084a259b314671a20522f9'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:56.667386 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1381/2726] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:25:56 2022] PUT /identity/v3/projects/71cd1bd1ecb846ddb8e14f6fe7dd9d64/users/bdb4c94cb50246e59a8c34708a4382df/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:25:56.678508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbbc52-10a5-490a-b4c1-58d02d1eceda None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:56.679653 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbbc52-10a5-490a-b4c1-58d02d1eceda None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:56.680440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbbc52-10a5-490a-b4c1-58d02d1eceda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:56.703426 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1346/2727] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:25:56 2022] DELETE /identity/v3/users/e4a3899ced084a259b314671a20522f9 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:56.755980 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-abcbbc52-10a5-490a-b4c1-58d02d1eceda None None] MFA Rules not processed for user `bdb4c94cb50246e59a8c34708a4382df`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:56.760042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-abcbbc52-10a5-490a-b4c1-58d02d1eceda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:56.872543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1382/2728] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:56.877222 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877c3a8-8a19-4339-85a7-5a81bb7b8c2a admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:56.878878 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a877c3a8-8a19-4339-85a7-5a81bb7b8c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:56.953734 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:56.984608 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877c3a8-8a19-4339-85a7-5a81bb7b8c2a admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:56.990584 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bbaf7-f7b0-4364-9015-773b3d3f2f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:56.991565 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bbaf7-f7b0-4364-9015-773b3d3f2f1f admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:56.992616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bbaf7-f7b0-4364-9015-773b3d3f2f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:56.993106 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bbaf7-f7b0-4364-9015-773b3d3f2f1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:56.994026 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bbaf7-f7b0-4364-9015-773b3d3f2f1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:56.996150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bbaf7-f7b0-4364-9015-773b3d3f2f1f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:57.110169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1347/2729] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:25:56 2022] POST /identity/v3/projects => generated 393 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:25:57.113899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcbbc52-10a5-490a-b4c1-58d02d1eceda None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:57.128091 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcbbc52-10a5-490a-b4c1-58d02d1eceda None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:57.245383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e24fa-b9f2-40e7-887f-e50a586e40c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:57.245383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e24fa-b9f2-40e7-887f-e50a586e40c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:57.245383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e24fa-b9f2-40e7-887f-e50a586e40c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:57.245383 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e24fa-b9f2-40e7-887f-e50a586e40c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:57.245900 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e24fa-b9f2-40e7-887f-e50a586e40c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:57.245900 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e24fa-b9f2-40e7-887f-e50a586e40c9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:57.389750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1383/2730] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:25:57 2022] POST /identity/v3/users => generated 303 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:25:57.392065 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bbaf7-f7b0-4364-9015-773b3d3f2f1f admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:57.404891 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e24fa-b9f2-40e7-887f-e50a586e40c9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:57.406370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bbaf7-f7b0-4364-9015-773b3d3f2f1f admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:57.407619 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7f0e24fa-b9f2-40e7-887f-e50a586e40c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:57.413103 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e56780-dd9a-4502-9994-bd942da023bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:57.413969 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e56780-dd9a-4502-9994-bd942da023bc admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:57.414664 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e56780-dd9a-4502-9994-bd942da023bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:57.415257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e56780-dd9a-4502-9994-bd942da023bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:57.420999 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e56780-dd9a-4502-9994-bd942da023bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:57.420999 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e56780-dd9a-4502-9994-bd942da023bc admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:57.421226 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e24fa-b9f2-40e7-887f-e50a586e40c9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:57.434049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1348/2731] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:25:57.435394 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e56780-dd9a-4502-9994-bd942da023bc admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:57.439598 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79ca3e-d3fd-4b98-b474-42b3f6328f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:57.440317 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79ca3e-d3fd-4b98-b474-42b3f6328f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:57.440598 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79ca3e-d3fd-4b98-b474-42b3f6328f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:57.442066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79ca3e-d3fd-4b98-b474-42b3f6328f1b admin admin] PATH_INFO: `/v3/projects/a4d690f0c2b84f12918387ef96321802` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:57.443933 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79ca3e-d3fd-4b98-b474-42b3f6328f1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4d690f0c2b84f12918387ef96321802)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:57.447241 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79ca3e-d3fd-4b98-b474-42b3f6328f1b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:57.449777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e56780-dd9a-4502-9994-bd942da023bc admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:57.455941 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3baef6c-7dde-449d-b2ab-aecdcafc56cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:57.456608 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3baef6c-7dde-449d-b2ab-aecdcafc56cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:57.456882 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3baef6c-7dde-449d-b2ab-aecdcafc56cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:57.457150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3baef6c-7dde-449d-b2ab-aecdcafc56cf admin admin] PATH_INFO: `/v3/projects/86c735b82ea743979e2774c72fc41c12/users/3cc70aaa43824f9f9e44f446d387f17b/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:57.480761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3baef6c-7dde-449d-b2ab-aecdcafc56cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=86c735b82ea743979e2774c72fc41c12, user_id=3cc70aaa43824f9f9e44f446d387f17b, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:57.483509 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3baef6c-7dde-449d-b2ab-aecdcafc56cf admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:57.589472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1349/2732] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:25:57 2022] PUT /identity/v3/projects/86c735b82ea743979e2774c72fc41c12/users/3cc70aaa43824f9f9e44f446d387f17b/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:57.592291 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3baef6c-7dde-449d-b2ab-aecdcafc56cf admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:57.660250 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ad79ca3e-d3fd-4b98-b474-42b3f6328f1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4d690f0c2b84f12918387ef96321802', 'request_id': 'req-ad79ca3e-d3fd-4b98-b474-42b3f6328f1b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:57.668474 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3baef6c-7dde-449d-b2ab-aecdcafc56cf admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:57.681345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ad79ca3e-d3fd-4b98-b474-42b3f6328f1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4d690f0c2b84f12918387ef96321802', 'request_id': 'req-ad79ca3e-d3fd-4b98-b474-42b3f6328f1b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:57.750116 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ad79ca3e-d3fd-4b98-b474-42b3f6328f1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4d690f0c2b84f12918387ef96321802', 'request_id': 'req-ad79ca3e-d3fd-4b98-b474-42b3f6328f1b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:57.777462 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ad79ca3e-d3fd-4b98-b474-42b3f6328f1b admin admin] The token cache is being invalidate because project a4d690f0c2b84f12918387ef96321802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:25:57.777922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ad79ca3e-d3fd-4b98-b474-42b3f6328f1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:57.780215 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ad79ca3e-d3fd-4b98-b474-42b3f6328f1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:25:57.784478 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1384/2733] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:25:57 2022] DELETE /identity/v3/projects/a4d690f0c2b84f12918387ef96321802 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:57.787734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f02b66-dabf-453e-87e2-ee34574259c3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:57.788243 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f02b66-dabf-453e-87e2-ee34574259c3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:57.789568 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f02b66-dabf-453e-87e2-ee34574259c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:57.814749 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d6f02b66-dabf-453e-87e2-ee34574259c3 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:57.825889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d6f02b66-dabf-453e-87e2-ee34574259c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:57.887780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:57.888393 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f370ff4-d643-45fb-ad0c-8a6218831f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:57.889246 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f370ff4-d643-45fb-ad0c-8a6218831f3d admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:57.889522 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f370ff4-d643-45fb-ad0c-8a6218831f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:57.889791 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f370ff4-d643-45fb-ad0c-8a6218831f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:57.890701 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f370ff4-d643-45fb-ad0c-8a6218831f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:57.892635 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f370ff4-d643-45fb-ad0c-8a6218831f3d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:57.904147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1350/2734] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:25:57.905553 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94792c3-987c-4799-b324-db628b4567e8 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:57.905838 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94792c3-987c-4799-b324-db628b4567e8 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:57.906111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94792c3-987c-4799-b324-db628b4567e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:57.937733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-e94792c3-987c-4799-b324-db628b4567e8 None None] MFA Rules not processed for user `79d29aa9482f4c1f9b1554333d85d80a`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:57.938242 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1385/2735] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:57.939274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e94792c3-987c-4799-b324-db628b4567e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:57.939425 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f02b66-dabf-453e-87e2-ee34574259c3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:57.940647 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d6f02b66-dabf-453e-87e2-ee34574259c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:57.952613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f02b66-dabf-453e-87e2-ee34574259c3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:57.958412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550420c0-f515-42c3-9d90-33ea2a5edd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:57.960397 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550420c0-f515-42c3-9d90-33ea2a5edd91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:57.960397 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550420c0-f515-42c3-9d90-33ea2a5edd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:57.960397 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550420c0-f515-42c3-9d90-33ea2a5edd91 admin admin] PATH_INFO: `/v3/projects/86c735b82ea743979e2774c72fc41c12/users/3cc70aaa43824f9f9e44f446d387f17b/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:57.963782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550420c0-f515-42c3-9d90-33ea2a5edd91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86c735b82ea743979e2774c72fc41c12, user_id=3cc70aaa43824f9f9e44f446d387f17b, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:57.966611 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550420c0-f515-42c3-9d90-33ea2a5edd91 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:57.977999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1386/2736] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:25:57 2022] PUT /identity/v3/projects/86c735b82ea743979e2774c72fc41c12/users/3cc70aaa43824f9f9e44f446d387f17b/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:57.980221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a88cf-5d8f-467f-81b7-0953856ad9d8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:57.980571 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a88cf-5d8f-467f-81b7-0953856ad9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:57.980945 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a88cf-5d8f-467f-81b7-0953856ad9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:58.002293 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-835a88cf-5d8f-467f-81b7-0953856ad9d8 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:58.006356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-835a88cf-5d8f-467f-81b7-0953856ad9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:58.016023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1351/2737] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:58.019598 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94792c3-987c-4799-b324-db628b4567e8 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:58.021238 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e94792c3-987c-4799-b324-db628b4567e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:58.027555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:58.042159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94792c3-987c-4799-b324-db628b4567e8 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:58.076978 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1387/2738] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:58.078277 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dda51-1f00-456d-bc62-020e011d3bf8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:58.078554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dda51-1f00-456d-bc62-020e011d3bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:58.078820 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dda51-1f00-456d-bc62-020e011d3bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:58.101356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824e34a-38bf-42f1-a303-4dcfc87aabf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:58.101910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824e34a-38bf-42f1-a303-4dcfc87aabf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:58.102186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824e34a-38bf-42f1-a303-4dcfc87aabf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:58.102447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824e34a-38bf-42f1-a303-4dcfc87aabf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:58.103097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4824e34a-38bf-42f1-a303-4dcfc87aabf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:58.105002 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4824e34a-38bf-42f1-a303-4dcfc87aabf0 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:58.107554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-414dda51-1f00-456d-bc62-020e011d3bf8 None None] MFA Rules not processed for user `3cc70aaa43824f9f9e44f446d387f17b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:58.108778 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-414dda51-1f00-456d-bc62-020e011d3bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:58.193037 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1388/2739] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:58 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:58.195708 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414dda51-1f00-456d-bc62-020e011d3bf8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:58.203628 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1352/2740] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:25:58 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:25:58.204456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824e34a-38bf-42f1-a303-4dcfc87aabf0 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:58.207851 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414dda51-1f00-456d-bc62-020e011d3bf8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:58.275101 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474739f-5c90-4ae2-8d88-0236f5d298ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:58.275875 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474739f-5c90-4ae2-8d88-0236f5d298ed admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:58.276158 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474739f-5c90-4ae2-8d88-0236f5d298ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:58.276566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474739f-5c90-4ae2-8d88-0236f5d298ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:58.277142 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e474739f-5c90-4ae2-8d88-0236f5d298ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:25:58.277445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e474739f-5c90-4ae2-8d88-0236f5d298ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:58.279247 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e474739f-5c90-4ae2-8d88-0236f5d298ed admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:58.281282 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824e34a-38bf-42f1-a303-4dcfc87aabf0 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:58.286094 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe17d8-2ba8-4a19-81d6-e9a006ad19e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:58.286695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe17d8-2ba8-4a19-81d6-e9a006ad19e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:58.286989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe17d8-2ba8-4a19-81d6-e9a006ad19e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:58.287263 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe17d8-2ba8-4a19-81d6-e9a006ad19e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:58.287887 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe17d8-2ba8-4a19-81d6-e9a006ad19e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:58.290839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe17d8-2ba8-4a19-81d6-e9a006ad19e8 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:58.292106 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1389/2741] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:25:58.299104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474739f-5c90-4ae2-8d88-0236f5d298ed admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:58.299104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e474739f-5c90-4ae2-8d88-0236f5d298ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:58.360214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474739f-5c90-4ae2-8d88-0236f5d298ed admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:58.364462 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcfeb5-fd2e-4199-9a69-d80f78834ab8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:58.365078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcfeb5-fd2e-4199-9a69-d80f78834ab8 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:58.365351 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcfeb5-fd2e-4199-9a69-d80f78834ab8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:58.365644 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcfeb5-fd2e-4199-9a69-d80f78834ab8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:58.411779 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1353/2742] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:25:58 2022] POST /identity/v3/users => generated 295 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:25:58.412731 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe17d8-2ba8-4a19-81d6-e9a006ad19e8 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:58.423351 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe17d8-2ba8-4a19-81d6-e9a006ad19e8 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:58.428057 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912302b-6a69-4f4e-8600-0904884d8a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:58.428652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912302b-6a69-4f4e-8600-0904884d8a3d admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:58.428934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912302b-6a69-4f4e-8600-0904884d8a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:58.429207 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912302b-6a69-4f4e-8600-0904884d8a3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:58.429858 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912302b-6a69-4f4e-8600-0904884d8a3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:58.432040 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912302b-6a69-4f4e-8600-0904884d8a3d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:58.434156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dcfeb5-fd2e-4199-9a69-d80f78834ab8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:58.435657 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dcfeb5-fd2e-4199-9a69-d80f78834ab8 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:58.446042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1390/2743] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:25:58 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:25:58.446956 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcfeb5-fd2e-4199-9a69-d80f78834ab8 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:58.454423 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:58.458563 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcfeb5-fd2e-4199-9a69-d80f78834ab8 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:58.466764 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daaf37a-f69a-4c24-a7a1-1ad5f8a9c613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:58.467496 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daaf37a-f69a-4c24-a7a1-1ad5f8a9c613 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:58.467960 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daaf37a-f69a-4c24-a7a1-1ad5f8a9c613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:58.468441 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daaf37a-f69a-4c24-a7a1-1ad5f8a9c613 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:58.469477 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daaf37a-f69a-4c24-a7a1-1ad5f8a9c613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:58.471541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daaf37a-f69a-4c24-a7a1-1ad5f8a9c613 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:58.485027 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1391/2744] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:25:58.488726 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daaf37a-f69a-4c24-a7a1-1ad5f8a9c613 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:58.498766 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daaf37a-f69a-4c24-a7a1-1ad5f8a9c613 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:58.510712 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520ec72-1b92-4616-b24b-c9046438e6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:58.510712 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520ec72-1b92-4616-b24b-c9046438e6cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:58.510712 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520ec72-1b92-4616-b24b-c9046438e6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:58.510712 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520ec72-1b92-4616-b24b-c9046438e6cd admin admin] PATH_INFO: `/v3/projects/d75a8b2600804cb593bcabaff5b0a4b7/users/99738b08189e43879f4ec96ea75fb82a/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:58.520710 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520ec72-1b92-4616-b24b-c9046438e6cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75a8b2600804cb593bcabaff5b0a4b7, user_id=99738b08189e43879f4ec96ea75fb82a, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:58.524067 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520ec72-1b92-4616-b24b-c9046438e6cd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:58.565762 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1354/2745] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:25:58 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:25:58.566696 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912302b-6a69-4f4e-8600-0904884d8a3d admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:58.568328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9912302b-6a69-4f4e-8600-0904884d8a3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:58.601633 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:58.631166 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912302b-6a69-4f4e-8600-0904884d8a3d admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:58.633801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af9549-7f69-494e-85c7-8ba5ae8db141 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:58.634404 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af9549-7f69-494e-85c7-8ba5ae8db141 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:58.634681 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af9549-7f69-494e-85c7-8ba5ae8db141 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:58.635092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af9549-7f69-494e-85c7-8ba5ae8db141 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:58.636891 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-12af9549-7f69-494e-85c7-8ba5ae8db141 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:58.662643 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1392/2746] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:25:58 2022] PUT /identity/v3/projects/d75a8b2600804cb593bcabaff5b0a4b7/users/99738b08189e43879f4ec96ea75fb82a/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:25:58.664431 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520ec72-1b92-4616-b24b-c9046438e6cd admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:58.701058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af9549-7f69-494e-85c7-8ba5ae8db141 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:58.702671 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af9549-7f69-494e-85c7-8ba5ae8db141 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:58.714438 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1355/2747] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:25:58 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:25:58.751236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af9549-7f69-494e-85c7-8ba5ae8db141 service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:58.752020 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520ec72-1b92-4616-b24b-c9046438e6cd admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:58.753824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0e346-5bf6-48ec-89f9-ea66784f5024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:58.755047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0e346-5bf6-48ec-89f9-ea66784f5024 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:58.755326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0e346-5bf6-48ec-89f9-ea66784f5024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:58.755652 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0e346-5bf6-48ec-89f9-ea66784f5024 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:58.756298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0e346-5bf6-48ec-89f9-ea66784f5024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:58.763066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0e346-5bf6-48ec-89f9-ea66784f5024 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:58.783621 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af9549-7f69-494e-85c7-8ba5ae8db141 service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:58.789312 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c6521-db47-482f-a563-2966422c872b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:58.790034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c6521-db47-482f-a563-2966422c872b admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:58.790406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c6521-db47-482f-a563-2966422c872b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:58.790783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c6521-db47-482f-a563-2966422c872b admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:58.791560 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c6521-db47-482f-a563-2966422c872b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:58.793272 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c6521-db47-482f-a563-2966422c872b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:58.805262 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1356/2748] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:25:58.809559 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c6521-db47-482f-a563-2966422c872b admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:58.821600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c6521-db47-482f-a563-2966422c872b admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:58.827798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb96b9-1f45-4729-8f11-8c2db0263fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:58.828498 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb96b9-1f45-4729-8f11-8c2db0263fa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:58.828784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb96b9-1f45-4729-8f11-8c2db0263fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:58.829124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb96b9-1f45-4729-8f11-8c2db0263fa9 admin admin] PATH_INFO: `/v3/projects/d75a8b2600804cb593bcabaff5b0a4b7/users/99738b08189e43879f4ec96ea75fb82a/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:58.831421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb96b9-1f45-4729-8f11-8c2db0263fa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75a8b2600804cb593bcabaff5b0a4b7, user_id=99738b08189e43879f4ec96ea75fb82a, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:58.834105 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb96b9-1f45-4729-8f11-8c2db0263fa9 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:58.850690 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1357/2749] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:25:58 2022] PUT /identity/v3/projects/d75a8b2600804cb593bcabaff5b0a4b7/users/99738b08189e43879f4ec96ea75fb82a/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:25:58.890532 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1393/2750] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:25:58 2022] POST /identity/v3/users => generated 294 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:25:58.891687 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb96b9-1f45-4729-8f11-8c2db0263fa9 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:58.956941 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb96b9-1f45-4729-8f11-8c2db0263fa9 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:58.963039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ac995-0f81-4a82-aea1-8c30760fa453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:58.963784 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ac995-0f81-4a82-aea1-8c30760fa453 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:58.964065 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ac995-0f81-4a82-aea1-8c30760fa453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:58.964335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ac995-0f81-4a82-aea1-8c30760fa453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:58.964985 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ac995-0f81-4a82-aea1-8c30760fa453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:58.966579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ac995-0f81-4a82-aea1-8c30760fa453 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:58.980806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1358/2751] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:25:58.981976 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0e346-5bf6-48ec-89f9-ea66784f5024 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:58.991874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0e346-5bf6-48ec-89f9-ea66784f5024 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:58.997350 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f65e9-61b1-4d70-ab0d-27e60c3f200e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:58.997954 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f65e9-61b1-4d70-ab0d-27e60c3f200e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:58.998231 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f65e9-61b1-4d70-ab0d-27e60c3f200e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:58.998526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f65e9-61b1-4d70-ab0d-27e60c3f200e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:58.999212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f65e9-61b1-4d70-ab0d-27e60c3f200e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:59.000808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f65e9-61b1-4d70-ab0d-27e60c3f200e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:59.011540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1394/2752] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:25:59.015977 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ac995-0f81-4a82-aea1-8c30760fa453 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:59.027492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ac995-0f81-4a82-aea1-8c30760fa453 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:59.034109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e33ba9-5469-494a-b8c3-61fd68f8e2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:59.034777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e33ba9-5469-494a-b8c3-61fd68f8e2ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:59.035085 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e33ba9-5469-494a-b8c3-61fd68f8e2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:59.035358 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e33ba9-5469-494a-b8c3-61fd68f8e2ad admin admin] PATH_INFO: `/v3/projects/a0735c7c56b44437a1986067966aefea/users/7b1417cb444644588a95707ac9c2abdf/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:59.052238 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e33ba9-5469-494a-b8c3-61fd68f8e2ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0735c7c56b44437a1986067966aefea, user_id=7b1417cb444644588a95707ac9c2abdf, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:59.055661 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e33ba9-5469-494a-b8c3-61fd68f8e2ad admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:59.165456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1359/2753] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:25:59 2022] PUT /identity/v3/projects/a0735c7c56b44437a1986067966aefea/users/7b1417cb444644588a95707ac9c2abdf/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:59.177001 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f859c-2c44-4081-b0dc-88a7a0428edf None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:59.177432 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f859c-2c44-4081-b0dc-88a7a0428edf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:59.177711 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f859c-2c44-4081-b0dc-88a7a0428edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:59.208302 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b10f859c-2c44-4081-b0dc-88a7a0428edf None None] MFA Rules not processed for user `7b1417cb444644588a95707ac9c2abdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:25:59.209761 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b10f859c-2c44-4081-b0dc-88a7a0428edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:25:59.307215 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1395/2754] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:25:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:25:59.308834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e33ba9-5469-494a-b8c3-61fd68f8e2ad admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:59.325616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:59.387156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e33ba9-5469-494a-b8c3-61fd68f8e2ad admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:59.392987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1377eec-e5d7-41d4-bbc5-34f8d13d8e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:59.393645 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1377eec-e5d7-41d4-bbc5-34f8d13d8e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:59.393928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1377eec-e5d7-41d4-bbc5-34f8d13d8e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:59.394194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1377eec-e5d7-41d4-bbc5-34f8d13d8e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:59.394872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1377eec-e5d7-41d4-bbc5-34f8d13d8e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:59.396827 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1377eec-e5d7-41d4-bbc5-34f8d13d8e5d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:59.494300 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:25:59.507011 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1360/2755] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:25:59 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:25:59.509908 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f859c-2c44-4081-b0dc-88a7a0428edf None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:59.529691 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f859c-2c44-4081-b0dc-88a7a0428edf None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:59.621033 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54c112-aab7-4612-8339-e24c6390b9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:59.622104 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54c112-aab7-4612-8339-e24c6390b9c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:59.622562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54c112-aab7-4612-8339-e24c6390b9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:59.623013 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54c112-aab7-4612-8339-e24c6390b9c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:59.624093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54c112-aab7-4612-8339-e24c6390b9c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:59.626640 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54c112-aab7-4612-8339-e24c6390b9c6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:59.707503 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1396/2756] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:25:59 2022] POST /identity/v3/users => generated 299 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:25:59.708846 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1377eec-e5d7-41d4-bbc5-34f8d13d8e5d admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:59.723811 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1377eec-e5d7-41d4-bbc5-34f8d13d8e5d admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:59.729994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5559d3-a243-43ac-b54b-e1a01825a570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:59.737419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5559d3-a243-43ac-b54b-e1a01825a570 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:59.737419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5559d3-a243-43ac-b54b-e1a01825a570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:59.737419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5559d3-a243-43ac-b54b-e1a01825a570 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:59.737419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5559d3-a243-43ac-b54b-e1a01825a570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:59.737419 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5559d3-a243-43ac-b54b-e1a01825a570 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:59.744818 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1361/2757] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:25:59.749041 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54c112-aab7-4612-8339-e24c6390b9c6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:59.759630 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54c112-aab7-4612-8339-e24c6390b9c6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:59.764331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c2499-481b-4a8e-96a5-ec0da88a5fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:59.765590 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c2499-481b-4a8e-96a5-ec0da88a5fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:59.765989 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c2499-481b-4a8e-96a5-ec0da88a5fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:59.766257 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c2499-481b-4a8e-96a5-ec0da88a5fd5 admin admin] PATH_INFO: `/v3/projects/08e9b69b25ce4df5b04b4cfa63bf6eec/users/c9f6a9265c7f465e9fdbe551215596bf/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:59.785513 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c2499-481b-4a8e-96a5-ec0da88a5fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e9b69b25ce4df5b04b4cfa63bf6eec, user_id=c9f6a9265c7f465e9fdbe551215596bf, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:59.790636 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c2499-481b-4a8e-96a5-ec0da88a5fd5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:59.889814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1397/2758] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:25:59 2022] PUT /identity/v3/projects/08e9b69b25ce4df5b04b4cfa63bf6eec/users/c9f6a9265c7f465e9fdbe551215596bf/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:25:59.894243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5559d3-a243-43ac-b54b-e1a01825a570 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:25:59.906241 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5559d3-a243-43ac-b54b-e1a01825a570 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:25:59.966254 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4585da-708d-4b61-9c1a-3dc24730445a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:25:59.967050 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4585da-708d-4b61-9c1a-3dc24730445a admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:25:59.967381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4585da-708d-4b61-9c1a-3dc24730445a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:25:59.967849 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4585da-708d-4b61-9c1a-3dc24730445a admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:25:59.968561 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4585da-708d-4b61-9c1a-3dc24730445a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:25:59.970265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4585da-708d-4b61-9c1a-3dc24730445a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:25:59.985747 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1362/2759] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:25:59.988445 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c2499-481b-4a8e-96a5-ec0da88a5fd5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:00.005399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c2499-481b-4a8e-96a5-ec0da88a5fd5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:00.011730 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b06a0-83f3-4487-b1e0-b41a7393f9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:00.012627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b06a0-83f3-4487-b1e0-b41a7393f9a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:00.013121 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b06a0-83f3-4487-b1e0-b41a7393f9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:00.013567 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b06a0-83f3-4487-b1e0-b41a7393f9a7 admin admin] PATH_INFO: `/v3/projects/08e9b69b25ce4df5b04b4cfa63bf6eec/users/c9f6a9265c7f465e9fdbe551215596bf/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:00.016097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b06a0-83f3-4487-b1e0-b41a7393f9a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e9b69b25ce4df5b04b4cfa63bf6eec, user_id=c9f6a9265c7f465e9fdbe551215596bf, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:00.019935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b06a0-83f3-4487-b1e0-b41a7393f9a7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:00.035709 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1398/2760] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:25:59 2022] PUT /identity/v3/projects/08e9b69b25ce4df5b04b4cfa63bf6eec/users/c9f6a9265c7f465e9fdbe551215596bf/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:00.046802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184c866-0207-4943-a9ae-8af5341c0900 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:00.047154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184c866-0207-4943-a9ae-8af5341c0900 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:00.047432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184c866-0207-4943-a9ae-8af5341c0900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:00.079544 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-5184c866-0207-4943-a9ae-8af5341c0900 None None] MFA Rules not processed for user `c9f6a9265c7f465e9fdbe551215596bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:00.080812 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5184c866-0207-4943-a9ae-8af5341c0900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:00.118228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:00.167434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1363/2761] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:00.184345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b06a0-83f3-4487-b1e0-b41a7393f9a7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:00.248986 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b06a0-83f3-4487-b1e0-b41a7393f9a7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:00.253221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bba989-21d0-4efa-a635-a121ac713aa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:00.253825 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bba989-21d0-4efa-a635-a121ac713aa2 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:00.254099 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bba989-21d0-4efa-a635-a121ac713aa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:00.254367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bba989-21d0-4efa-a635-a121ac713aa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:00.265220 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bba989-21d0-4efa-a635-a121ac713aa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:00.266847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bba989-21d0-4efa-a635-a121ac713aa2 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:00.280450 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1399/2762] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:26:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:26:00.350869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184c866-0207-4943-a9ae-8af5341c0900 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:00.375962 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:00.432338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184c866-0207-4943-a9ae-8af5341c0900 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:00.438716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00620ee9-2026-454e-a664-1e6d405c3ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:00.439976 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00620ee9-2026-454e-a664-1e6d405c3ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:00.440258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00620ee9-2026-454e-a664-1e6d405c3ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:00.440534 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00620ee9-2026-454e-a664-1e6d405c3ece admin admin] PATH_INFO: `/v3/users/7b1417cb444644588a95707ac9c2abdf` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:00.442135 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00620ee9-2026-454e-a664-1e6d405c3ece admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b1417cb444644588a95707ac9c2abdf)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:00.449628 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00620ee9-2026-454e-a664-1e6d405c3ece admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:00.731219 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-00620ee9-2026-454e-a664-1e6d405c3ece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b1417cb444644588a95707ac9c2abdf'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:00.731219 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-00620ee9-2026-454e-a664-1e6d405c3ece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b1417cb444644588a95707ac9c2abdf'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:00.758319 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-00620ee9-2026-454e-a664-1e6d405c3ece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b1417cb444644588a95707ac9c2abdf'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:00.831088 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-00620ee9-2026-454e-a664-1e6d405c3ece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b1417cb444644588a95707ac9c2abdf'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:00.831706 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-00620ee9-2026-454e-a664-1e6d405c3ece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b1417cb444644588a95707ac9c2abdf'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:00.851462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1364/2763] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:26:00 2022] DELETE /identity/v3/users/7b1417cb444644588a95707ac9c2abdf => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:00.856656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6300b1-51cb-472c-9ead-967c83f5ff1a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:00.857183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6300b1-51cb-472c-9ead-967c83f5ff1a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:00.857436 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6300b1-51cb-472c-9ead-967c83f5ff1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:00.897583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fe6300b1-51cb-472c-9ead-967c83f5ff1a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:00.900119 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fe6300b1-51cb-472c-9ead-967c83f5ff1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:00.988838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1400/2764] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:00.998722 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00620ee9-2026-454e-a664-1e6d405c3ece admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:01.000463 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-00620ee9-2026-454e-a664-1e6d405c3ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:01.082048 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00620ee9-2026-454e-a664-1e6d405c3ece admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:01.087046 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ef508-81c8-4e75-83f9-f5b6e3b25b23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:01.091652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ef508-81c8-4e75-83f9-f5b6e3b25b23 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:01.091652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ef508-81c8-4e75-83f9-f5b6e3b25b23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:01.091652 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ef508-81c8-4e75-83f9-f5b6e3b25b23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:01.100744 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ef508-81c8-4e75-83f9-f5b6e3b25b23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:01.102903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ef508-81c8-4e75-83f9-f5b6e3b25b23 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:01.114517 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1365/2765] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:26:01.222843 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:01.971961 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6300b1-51cb-472c-9ead-967c83f5ff1a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:01.975037 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fe6300b1-51cb-472c-9ead-967c83f5ff1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:02.010246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6300b1-51cb-472c-9ead-967c83f5ff1a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:02.022815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2094d-0033-4d9f-9daf-dae83a4d4154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:02.024406 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2094d-0033-4d9f-9daf-dae83a4d4154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:02.024687 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2094d-0033-4d9f-9daf-dae83a4d4154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:02.024965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2094d-0033-4d9f-9daf-dae83a4d4154 admin admin] PATH_INFO: `/v3/projects/a0735c7c56b44437a1986067966aefea` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:02.031893 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2094d-0033-4d9f-9daf-dae83a4d4154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0735c7c56b44437a1986067966aefea)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:02.034807 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2094d-0033-4d9f-9daf-dae83a4d4154 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:02.203083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f7a2094d-0033-4d9f-9daf-dae83a4d4154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0735c7c56b44437a1986067966aefea', 'request_id': 'req-f7a2094d-0033-4d9f-9daf-dae83a4d4154'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:02.211083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f7a2094d-0033-4d9f-9daf-dae83a4d4154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0735c7c56b44437a1986067966aefea', 'request_id': 'req-f7a2094d-0033-4d9f-9daf-dae83a4d4154'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:02.331088 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f7a2094d-0033-4d9f-9daf-dae83a4d4154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0735c7c56b44437a1986067966aefea', 'request_id': 'req-f7a2094d-0033-4d9f-9daf-dae83a4d4154'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:02.345215 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:02.349795 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f7a2094d-0033-4d9f-9daf-dae83a4d4154 admin admin] The token cache is being invalidate because project a0735c7c56b44437a1986067966aefea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:26:02.350343 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f7a2094d-0033-4d9f-9daf-dae83a4d4154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:02.355586 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f7a2094d-0033-4d9f-9daf-dae83a4d4154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:02.357479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1401/2766] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:26:01 2022] DELETE /identity/v3/projects/a0735c7c56b44437a1986067966aefea => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:26:02.361527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ef508-81c8-4e75-83f9-f5b6e3b25b23 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:02.378073 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ef508-81c8-4e75-83f9-f5b6e3b25b23 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:02.482096 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6e330-2358-4014-8452-4ada09aefda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:02.483622 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6e330-2358-4014-8452-4ada09aefda0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:02.484036 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6e330-2358-4014-8452-4ada09aefda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:02.484340 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6e330-2358-4014-8452-4ada09aefda0 admin admin] PATH_INFO: `/v3/users/c9f6a9265c7f465e9fdbe551215596bf` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:02.486317 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb6e330-2358-4014-8452-4ada09aefda0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9f6a9265c7f465e9fdbe551215596bf)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:02.488839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb6e330-2358-4014-8452-4ada09aefda0 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:02.693521 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-ccb6e330-2358-4014-8452-4ada09aefda0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9f6a9265c7f465e9fdbe551215596bf'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:02.694387 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-ccb6e330-2358-4014-8452-4ada09aefda0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9f6a9265c7f465e9fdbe551215596bf'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:02.713631 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-ccb6e330-2358-4014-8452-4ada09aefda0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9f6a9265c7f465e9fdbe551215596bf'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:02.811629 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-ccb6e330-2358-4014-8452-4ada09aefda0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9f6a9265c7f465e9fdbe551215596bf'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:02.813406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-ccb6e330-2358-4014-8452-4ada09aefda0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9f6a9265c7f465e9fdbe551215596bf'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:02.837441 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1366/2767] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:26:02 2022] DELETE /identity/v3/users/c9f6a9265c7f465e9fdbe551215596bf => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:03.336716 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2094d-0033-4d9f-9daf-dae83a4d4154 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:03.339434 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f7a2094d-0033-4d9f-9daf-dae83a4d4154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:03.365608 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2094d-0033-4d9f-9daf-dae83a4d4154 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:03.456215 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2de7ee-bad3-4d94-8bf5-64d74946776f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:03.456895 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2de7ee-bad3-4d94-8bf5-64d74946776f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:03.457168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2de7ee-bad3-4d94-8bf5-64d74946776f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:03.457437 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2de7ee-bad3-4d94-8bf5-64d74946776f admin admin] PATH_INFO: `/v3/projects/08e9b69b25ce4df5b04b4cfa63bf6eec` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:03.458562 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2de7ee-bad3-4d94-8bf5-64d74946776f admin admin] RBAC: Authorizing `identity:delete_project(project_id=08e9b69b25ce4df5b04b4cfa63bf6eec)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:03.460902 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2de7ee-bad3-4d94-8bf5-64d74946776f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:03.575714 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6d2de7ee-bad3-4d94-8bf5-64d74946776f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08e9b69b25ce4df5b04b4cfa63bf6eec', 'request_id': 'req-6d2de7ee-bad3-4d94-8bf5-64d74946776f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:03.583097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6d2de7ee-bad3-4d94-8bf5-64d74946776f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08e9b69b25ce4df5b04b4cfa63bf6eec', 'request_id': 'req-6d2de7ee-bad3-4d94-8bf5-64d74946776f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:03.637140 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6d2de7ee-bad3-4d94-8bf5-64d74946776f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08e9b69b25ce4df5b04b4cfa63bf6eec', 'request_id': 'req-6d2de7ee-bad3-4d94-8bf5-64d74946776f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:03.656433 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6d2de7ee-bad3-4d94-8bf5-64d74946776f admin admin] The token cache is being invalidate because project 08e9b69b25ce4df5b04b4cfa63bf6eec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:26:03.656723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6d2de7ee-bad3-4d94-8bf5-64d74946776f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:03.657727 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6d2de7ee-bad3-4d94-8bf5-64d74946776f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:03.659851 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1402/2768] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:26:03 2022] DELETE /identity/v3/projects/08e9b69b25ce4df5b04b4cfa63bf6eec => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:03.668007 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d77fb7-4470-40ee-b7d3-8f09b1324806 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:03.669711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d77fb7-4470-40ee-b7d3-8f09b1324806 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:03.670203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d77fb7-4470-40ee-b7d3-8f09b1324806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:03.697666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-d2d77fb7-4470-40ee-b7d3-8f09b1324806 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:03.699562 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d2d77fb7-4470-40ee-b7d3-8f09b1324806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:03.776629 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1367/2769] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:03.785135 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f431d8-39eb-4b01-9f4e-406a87342911 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:03.785442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f431d8-39eb-4b01-9f4e-406a87342911 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:03.785674 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f431d8-39eb-4b01-9f4e-406a87342911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:03.807599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-29f431d8-39eb-4b01-9f4e-406a87342911 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:03.808806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-29f431d8-39eb-4b01-9f4e-406a87342911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:03.821058 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1403/2770] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:03.821795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d77fb7-4470-40ee-b7d3-8f09b1324806 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:03.832836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d77fb7-4470-40ee-b7d3-8f09b1324806 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:03.838091 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f8ac7-8659-4ce8-b46d-f1faf58a157c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:03.838668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f8ac7-8659-4ce8-b46d-f1faf58a157c admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:03.838960 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f8ac7-8659-4ce8-b46d-f1faf58a157c admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:03.839234 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f8ac7-8659-4ce8-b46d-f1faf58a157c admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:03.840040 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f8ac7-8659-4ce8-b46d-f1faf58a157c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:26:03.840182 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f8ac7-8659-4ce8-b46d-f1faf58a157c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:03.841911 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f8ac7-8659-4ce8-b46d-f1faf58a157c admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:03.851694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:03.855335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1368/2771] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:26:03.859848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f431d8-39eb-4b01-9f4e-406a87342911 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:03.869679 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f431d8-39eb-4b01-9f4e-406a87342911 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:03.875209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06bc46-a1a4-471e-b449-37a60f69c2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:03.875820 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06bc46-a1a4-471e-b449-37a60f69c2a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:03.876100 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06bc46-a1a4-471e-b449-37a60f69c2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:03.876360 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06bc46-a1a4-471e-b449-37a60f69c2a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:03.877006 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06bc46-a1a4-471e-b449-37a60f69c2a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:03.878920 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06bc46-a1a4-471e-b449-37a60f69c2a5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:04.011180 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1404/2772] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:26:03 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:26:04.015716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f8ac7-8659-4ce8-b46d-f1faf58a157c admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:04.026695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f8ac7-8659-4ce8-b46d-f1faf58a157c admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:04.089802 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0911ec0-4830-4da8-b859-5eaeb766ffd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:04.090515 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0911ec0-4830-4da8-b859-5eaeb766ffd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:04.090873 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0911ec0-4830-4da8-b859-5eaeb766ffd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:04.091286 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0911ec0-4830-4da8-b859-5eaeb766ffd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:04.092004 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0911ec0-4830-4da8-b859-5eaeb766ffd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:04.094068 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0911ec0-4830-4da8-b859-5eaeb766ffd5 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:04.206980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1369/2773] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:26:04 2022] POST /identity/v3/users => generated 294 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:26:04.209703 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06bc46-a1a4-471e-b449-37a60f69c2a5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:04.224627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06bc46-a1a4-471e-b449-37a60f69c2a5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:04.231548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8da2f-102e-460b-898a-7de4024d8bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:04.232157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8da2f-102e-460b-898a-7de4024d8bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:04.232915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8da2f-102e-460b-898a-7de4024d8bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:04.233342 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8da2f-102e-460b-898a-7de4024d8bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:04.233992 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8da2f-102e-460b-898a-7de4024d8bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:04.235748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8da2f-102e-460b-898a-7de4024d8bc5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:04.248412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1405/2774] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:26:04.252810 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0911ec0-4830-4da8-b859-5eaeb766ffd5 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:04.266655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0911ec0-4830-4da8-b859-5eaeb766ffd5 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:04.276034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c34ba8-5e3a-482f-b057-12c6c80a1079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:04.276034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c34ba8-5e3a-482f-b057-12c6c80a1079 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:04.276034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c34ba8-5e3a-482f-b057-12c6c80a1079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:04.276034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c34ba8-5e3a-482f-b057-12c6c80a1079 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:04.276411 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c34ba8-5e3a-482f-b057-12c6c80a1079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:04.277241 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c34ba8-5e3a-482f-b057-12c6c80a1079 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:04.288309 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1370/2775] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:26:04.292839 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8da2f-102e-460b-898a-7de4024d8bc5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:04.302883 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8da2f-102e-460b-898a-7de4024d8bc5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:04.310437 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6072ec5-fa49-4c15-a03c-f4aa275a1bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:04.311145 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6072ec5-fa49-4c15-a03c-f4aa275a1bcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:04.311426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6072ec5-fa49-4c15-a03c-f4aa275a1bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:04.311699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6072ec5-fa49-4c15-a03c-f4aa275a1bcc admin admin] PATH_INFO: `/v3/projects/96d123f1bcdb4a10a756f01a2e1d5746/users/97b2b03113c7495d996dbe062bb3bfde/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:04.327553 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6072ec5-fa49-4c15-a03c-f4aa275a1bcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=96d123f1bcdb4a10a756f01a2e1d5746, user_id=97b2b03113c7495d996dbe062bb3bfde, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:04.330234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6072ec5-fa49-4c15-a03c-f4aa275a1bcc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:04.381972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c34ba8-5e3a-482f-b057-12c6c80a1079 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:04.384755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f5c34ba8-5e3a-482f-b057-12c6c80a1079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:04.399236 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c34ba8-5e3a-482f-b057-12c6c80a1079 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:04.405607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb9cc9-3d99-40aa-a44e-ab0444d75d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:04.406319 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb9cc9-3d99-40aa-a44e-ab0444d75d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:04.406691 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb9cc9-3d99-40aa-a44e-ab0444d75d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:04.407062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb9cc9-3d99-40aa-a44e-ab0444d75d9f admin admin] PATH_INFO: `/v3/users/bdb4c94cb50246e59a8c34708a4382df` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:04.408576 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb9cc9-3d99-40aa-a44e-ab0444d75d9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdb4c94cb50246e59a8c34708a4382df)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:04.410302 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb9cc9-3d99-40aa-a44e-ab0444d75d9f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:04.524060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1406/2776] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:26:04 2022] PUT /identity/v3/projects/96d123f1bcdb4a10a756f01a2e1d5746/users/97b2b03113c7495d996dbe062bb3bfde/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:04.537872 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858b113-7ce2-416d-afc8-009e44d34774 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:04.538237 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858b113-7ce2-416d-afc8-009e44d34774 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:04.539225 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858b113-7ce2-416d-afc8-009e44d34774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:04.611153 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b858b113-7ce2-416d-afc8-009e44d34774 None None] MFA Rules not processed for user `97b2b03113c7495d996dbe062bb3bfde`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:04.612404 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b858b113-7ce2-416d-afc8-009e44d34774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:04.637842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:04.685345 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-14eb9cc9-3d99-40aa-a44e-ab0444d75d9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdb4c94cb50246e59a8c34708a4382df'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:04.686468 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-14eb9cc9-3d99-40aa-a44e-ab0444d75d9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdb4c94cb50246e59a8c34708a4382df'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:04.710574 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-14eb9cc9-3d99-40aa-a44e-ab0444d75d9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdb4c94cb50246e59a8c34708a4382df'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:04.721912 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1407/2777] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:04.728183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858b113-7ce2-416d-afc8-009e44d34774 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:04.729916 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b858b113-7ce2-416d-afc8-009e44d34774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:04.812843 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858b113-7ce2-416d-afc8-009e44d34774 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:04.818437 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fe470-a466-4319-95f3-110df65fd0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:04.819087 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fe470-a466-4319-95f3-110df65fd0f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:04.819356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fe470-a466-4319-95f3-110df65fd0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:04.819633 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fe470-a466-4319-95f3-110df65fd0f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:04.820720 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563fe470-a466-4319-95f3-110df65fd0f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:04.822026 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-14eb9cc9-3d99-40aa-a44e-ab0444d75d9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdb4c94cb50246e59a8c34708a4382df'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:04.822836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-14eb9cc9-3d99-40aa-a44e-ab0444d75d9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdb4c94cb50246e59a8c34708a4382df'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:04.823354 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563fe470-a466-4319-95f3-110df65fd0f5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:04.841018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1371/2778] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:26:04 2022] DELETE /identity/v3/users/bdb4c94cb50246e59a8c34708a4382df => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:26:04.846001 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab77b07-9de4-41fd-8102-5a4bfa3fa17e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:04.846737 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab77b07-9de4-41fd-8102-5a4bfa3fa17e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:04.847255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab77b07-9de4-41fd-8102-5a4bfa3fa17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:04.872516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-1ab77b07-9de4-41fd-8102-5a4bfa3fa17e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:04.872516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1ab77b07-9de4-41fd-8102-5a4bfa3fa17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:04.924328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:04.925135 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1408/2779] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:26:04 2022] POST /identity/v3/projects => generated 383 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:26:04.926500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fe470-a466-4319-95f3-110df65fd0f5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:04.946454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1372/2780] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:04.957335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab77b07-9de4-41fd-8102-5a4bfa3fa17e None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:04.958872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1ab77b07-9de4-41fd-8102-5a4bfa3fa17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:05.039297 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fe470-a466-4319-95f3-110df65fd0f5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:05.040179 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab77b07-9de4-41fd-8102-5a4bfa3fa17e None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:05.043865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16988c10-6b50-4661-98b7-8de19f824c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:05.044672 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16988c10-6b50-4661-98b7-8de19f824c69 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:05.044944 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16988c10-6b50-4661-98b7-8de19f824c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:05.045210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16988c10-6b50-4661-98b7-8de19f824c69 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:05.045847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16988c10-6b50-4661-98b7-8de19f824c69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:05.047811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16988c10-6b50-4661-98b7-8de19f824c69 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:05.106307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83dec1-81c5-42df-86c8-ac5bd2340d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:05.106928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83dec1-81c5-42df-86c8-ac5bd2340d46 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:05.107243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83dec1-81c5-42df-86c8-ac5bd2340d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:05.107515 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83dec1-81c5-42df-86c8-ac5bd2340d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:05.123525 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83dec1-81c5-42df-86c8-ac5bd2340d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:05.125640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83dec1-81c5-42df-86c8-ac5bd2340d46 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:05.137690 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1409/2781] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:26:04 2022] POST /identity/v3/users => generated 298 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:26:05.139741 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1373/2782] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:26:05.143515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16988c10-6b50-4661-98b7-8de19f824c69 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:05.160320 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16988c10-6b50-4661-98b7-8de19f824c69 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:05.166844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029521c3-f4ac-476b-82c5-e61109ef4a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:05.167543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029521c3-f4ac-476b-82c5-e61109ef4a51 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:05.167814 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029521c3-f4ac-476b-82c5-e61109ef4a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:05.168085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029521c3-f4ac-476b-82c5-e61109ef4a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:05.168734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029521c3-f4ac-476b-82c5-e61109ef4a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:05.170345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029521c3-f4ac-476b-82c5-e61109ef4a51 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:05.186075 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1410/2783] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:26:05.190872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83dec1-81c5-42df-86c8-ac5bd2340d46 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:05.204663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83dec1-81c5-42df-86c8-ac5bd2340d46 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:05.214054 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547d579-5005-4807-aeb6-f3efdd37eb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:05.214978 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547d579-5005-4807-aeb6-f3efdd37eb89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:05.215401 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547d579-5005-4807-aeb6-f3efdd37eb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:05.215777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547d579-5005-4807-aeb6-f3efdd37eb89 admin admin] PATH_INFO: `/v3/projects/5324f53294754decb9e192a6f9ae2950/users/c74f13a0885a4a4685c45e821987b301/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:05.238621 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f547d579-5005-4807-aeb6-f3efdd37eb89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5324f53294754decb9e192a6f9ae2950, user_id=c74f13a0885a4a4685c45e821987b301, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:05.243563 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f547d579-5005-4807-aeb6-f3efdd37eb89 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:05.308536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1374/2784] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:26:05 2022] PUT /identity/v3/projects/5324f53294754decb9e192a6f9ae2950/users/c74f13a0885a4a4685c45e821987b301/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:26:05.312985 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029521c3-f4ac-476b-82c5-e61109ef4a51 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:05.325610 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029521c3-f4ac-476b-82c5-e61109ef4a51 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:05.431755 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:05.443002 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fc7a2-b3ed-433a-b6a7-0b5315b0e562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:05.443633 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fc7a2-b3ed-433a-b6a7-0b5315b0e562 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:05.444384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fc7a2-b3ed-433a-b6a7-0b5315b0e562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:05.444384 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fc7a2-b3ed-433a-b6a7-0b5315b0e562 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:05.445396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fc7a2-b3ed-433a-b6a7-0b5315b0e562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:05.447085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fc7a2-b3ed-433a-b6a7-0b5315b0e562 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:05.462613 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1411/2785] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:26:05.464117 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547d579-5005-4807-aeb6-f3efdd37eb89 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:05.477041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547d579-5005-4807-aeb6-f3efdd37eb89 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:05.482550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91610376-f8f9-4113-bce0-af8a5903e44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:05.483367 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91610376-f8f9-4113-bce0-af8a5903e44b admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:05.483713 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91610376-f8f9-4113-bce0-af8a5903e44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:05.484052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91610376-f8f9-4113-bce0-af8a5903e44b admin admin] PATH_INFO: `/v3/projects/5324f53294754decb9e192a6f9ae2950/users/c74f13a0885a4a4685c45e821987b301/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:05.487984 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91610376-f8f9-4113-bce0-af8a5903e44b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5324f53294754decb9e192a6f9ae2950, user_id=c74f13a0885a4a4685c45e821987b301, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:05.491047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91610376-f8f9-4113-bce0-af8a5903e44b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:05.520659 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1375/2786] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:26:05 2022] PUT /identity/v3/projects/5324f53294754decb9e192a6f9ae2950/users/c74f13a0885a4a4685c45e821987b301/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:26:05.522792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba20d4-81ef-42fc-831e-fc3c79af0143 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:05.523657 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba20d4-81ef-42fc-831e-fc3c79af0143 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:05.523930 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba20d4-81ef-42fc-831e-fc3c79af0143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:05.590760 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c7ba20d4-81ef-42fc-831e-fc3c79af0143 None None] MFA Rules not processed for user `c74f13a0885a4a4685c45e821987b301`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:05.592059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c7ba20d4-81ef-42fc-831e-fc3c79af0143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:05.649835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91610376-f8f9-4113-bce0-af8a5903e44b admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:05.651683 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-91610376-f8f9-4113-bce0-af8a5903e44b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:05.707660 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1412/2787] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:26:05.720527 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604cc4f6-92d8-49aa-ab7e-238db639f8a0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:05.721331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604cc4f6-92d8-49aa-ab7e-238db639f8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:05.721331 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604cc4f6-92d8-49aa-ab7e-238db639f8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:05.763083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-604cc4f6-92d8-49aa-ab7e-238db639f8a0 None None] MFA Rules not processed for user `99738b08189e43879f4ec96ea75fb82a`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:05.764358 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-604cc4f6-92d8-49aa-ab7e-238db639f8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:05.774290 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91610376-f8f9-4113-bce0-af8a5903e44b admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:05.779856 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4bede-daca-435a-aef1-ec2096854b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:05.780499 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4bede-daca-435a-aef1-ec2096854b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:05.780931 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4bede-daca-435a-aef1-ec2096854b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:05.781206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4bede-daca-435a-aef1-ec2096854b47 admin admin] PATH_INFO: `/v3/projects/71cd1bd1ecb846ddb8e14f6fe7dd9d64` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:05.782382 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4bede-daca-435a-aef1-ec2096854b47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71cd1bd1ecb846ddb8e14f6fe7dd9d64)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:05.784346 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4bede-daca-435a-aef1-ec2096854b47 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:05.802906 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:05.844543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1413/2788] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:26:05.845439 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604cc4f6-92d8-49aa-ab7e-238db639f8a0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:05.847210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-604cc4f6-92d8-49aa-ab7e-238db639f8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:05.885809 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:05.900148 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6bf4bede-daca-435a-aef1-ec2096854b47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71cd1bd1ecb846ddb8e14f6fe7dd9d64', 'request_id': 'req-6bf4bede-daca-435a-aef1-ec2096854b47'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:05.906713 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6bf4bede-daca-435a-aef1-ec2096854b47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71cd1bd1ecb846ddb8e14f6fe7dd9d64', 'request_id': 'req-6bf4bede-daca-435a-aef1-ec2096854b47'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:05.913312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604cc4f6-92d8-49aa-ab7e-238db639f8a0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:05.963700 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08c11b-49fa-4223-a499-d7629fe7030b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:05.964353 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08c11b-49fa-4223-a499-d7629fe7030b service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:05.964806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08c11b-49fa-4223-a499-d7629fe7030b service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:05.965080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08c11b-49fa-4223-a499-d7629fe7030b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:05.987107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6bf4bede-daca-435a-aef1-ec2096854b47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71cd1bd1ecb846ddb8e14f6fe7dd9d64', 'request_id': 'req-6bf4bede-daca-435a-aef1-ec2096854b47'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:06.000623 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6bf4bede-daca-435a-aef1-ec2096854b47 admin admin] The token cache is being invalidate because project 71cd1bd1ecb846ddb8e14f6fe7dd9d64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:26:06.001016 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6bf4bede-daca-435a-aef1-ec2096854b47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:06.001739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-6bf4bede-daca-435a-aef1-ec2096854b47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:06.003589 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1376/2789] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:26:05 2022] DELETE /identity/v3/projects/71cd1bd1ecb846ddb8e14f6fe7dd9d64 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:26:06.004555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4bede-daca-435a-aef1-ec2096854b47 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:06.006036 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6bf4bede-daca-435a-aef1-ec2096854b47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:06.030304 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08c11b-49fa-4223-a499-d7629fe7030b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:06.031765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08c11b-49fa-4223-a499-d7629fe7030b service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:06.042928 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1414/2790] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:26:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:26:06.043840 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08c11b-49fa-4223-a499-d7629fe7030b service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:06.093164 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4bede-daca-435a-aef1-ec2096854b47 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:06.099978 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73ceb6-281a-4103-b54b-f9a4a9259d26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:06.102293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73ceb6-281a-4103-b54b-f9a4a9259d26 service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:06.102293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73ceb6-281a-4103-b54b-f9a4a9259d26 service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:06.102293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73ceb6-281a-4103-b54b-f9a4a9259d26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:06.104194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9a73ceb6-281a-4103-b54b-f9a4a9259d26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:06.113188 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08c11b-49fa-4223-a499-d7629fe7030b service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:06.120385 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20b073-c0a0-4f75-958c-0321d24a2f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:06.121018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20b073-c0a0-4f75-958c-0321d24a2f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:06.121290 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20b073-c0a0-4f75-958c-0321d24a2f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:06.121558 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20b073-c0a0-4f75-958c-0321d24a2f24 admin admin] PATH_INFO: `/v3/users/3cc70aaa43824f9f9e44f446d387f17b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:06.122726 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20b073-c0a0-4f75-958c-0321d24a2f24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cc70aaa43824f9f9e44f446d387f17b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:06.124685 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20b073-c0a0-4f75-958c-0321d24a2f24 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:06.190269 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73ceb6-281a-4103-b54b-f9a4a9259d26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:06.191940 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73ceb6-281a-4103-b54b-f9a4a9259d26 service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:06.205202 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1377/2791] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:26:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:26:06.318136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:06.328624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6f20b073-c0a0-4f75-958c-0321d24a2f24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cc70aaa43824f9f9e44f446d387f17b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:06.346963 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6f20b073-c0a0-4f75-958c-0321d24a2f24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cc70aaa43824f9f9e44f446d387f17b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:06.347777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6f20b073-c0a0-4f75-958c-0321d24a2f24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cc70aaa43824f9f9e44f446d387f17b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:06.434011 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6f20b073-c0a0-4f75-958c-0321d24a2f24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cc70aaa43824f9f9e44f446d387f17b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:06.442801 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6f20b073-c0a0-4f75-958c-0321d24a2f24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cc70aaa43824f9f9e44f446d387f17b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:06.478805 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1415/2792] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:26:06 2022] DELETE /identity/v3/users/3cc70aaa43824f9f9e44f446d387f17b => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:06.949497 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73ceb6-281a-4103-b54b-f9a4a9259d26 service nova] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:06.951022 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9a73ceb6-281a-4103-b54b-f9a4a9259d26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:06.972239 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73ceb6-281a-4103-b54b-f9a4a9259d26 service nova] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:07.013101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:07.032144 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2711b-32e2-42d4-9cbf-1caf25473e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:07.032773 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2711b-32e2-42d4-9cbf-1caf25473e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:07.033047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2711b-32e2-42d4-9cbf-1caf25473e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:07.033314 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2711b-32e2-42d4-9cbf-1caf25473e5a admin admin] PATH_INFO: `/v3/projects/86c735b82ea743979e2774c72fc41c12` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:07.034704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2711b-32e2-42d4-9cbf-1caf25473e5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=86c735b82ea743979e2774c72fc41c12)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:07.036689 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2711b-32e2-42d4-9cbf-1caf25473e5a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:07.175111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e5a2711b-32e2-42d4-9cbf-1caf25473e5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86c735b82ea743979e2774c72fc41c12', 'request_id': 'req-e5a2711b-32e2-42d4-9cbf-1caf25473e5a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:07.181867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e5a2711b-32e2-42d4-9cbf-1caf25473e5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86c735b82ea743979e2774c72fc41c12', 'request_id': 'req-e5a2711b-32e2-42d4-9cbf-1caf25473e5a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:07.234069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20b073-c0a0-4f75-958c-0321d24a2f24 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:07.235529 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6f20b073-c0a0-4f75-958c-0321d24a2f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:07.250224 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20b073-c0a0-4f75-958c-0321d24a2f24 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:07.273228 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e5a2711b-32e2-42d4-9cbf-1caf25473e5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86c735b82ea743979e2774c72fc41c12', 'request_id': 'req-e5a2711b-32e2-42d4-9cbf-1caf25473e5a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:07.288106 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e5a2711b-32e2-42d4-9cbf-1caf25473e5a admin admin] The token cache is being invalidate because project 86c735b82ea743979e2774c72fc41c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:26:07.288607 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e5a2711b-32e2-42d4-9cbf-1caf25473e5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:07.289495 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-e5a2711b-32e2-42d4-9cbf-1caf25473e5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:07.296823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1378/2793] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:26:06 2022] DELETE /identity/v3/projects/86c735b82ea743979e2774c72fc41c12 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:07.298754 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5947de-fb79-408f-8575-b9ad7427ca9f None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:07.299289 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5947de-fb79-408f-8575-b9ad7427ca9f None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:07.300251 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5947de-fb79-408f-8575-b9ad7427ca9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:07.317017 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0011d3-e563-43f1-8cb2-2195b28357da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:07.317734 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0011d3-e563-43f1-8cb2-2195b28357da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:07.318079 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0011d3-e563-43f1-8cb2-2195b28357da admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:07.318425 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0011d3-e563-43f1-8cb2-2195b28357da admin admin] PATH_INFO: `/v3/users/97b2b03113c7495d996dbe062bb3bfde` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:07.320324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0011d3-e563-43f1-8cb2-2195b28357da admin admin] RBAC: Authorizing `identity:delete_user(user_id=97b2b03113c7495d996dbe062bb3bfde)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:07.322372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0011d3-e563-43f1-8cb2-2195b28357da admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:07.323588 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-9b5947de-fb79-408f-8575-b9ad7427ca9f None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:07.325392 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9b5947de-fb79-408f-8575-b9ad7427ca9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:07.336361 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1379/2794] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:07.343672 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b47212-a38f-4c8d-8ae9-c670e484e976 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:07.343981 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b47212-a38f-4c8d-8ae9-c670e484e976 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:07.344254 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b47212-a38f-4c8d-8ae9-c670e484e976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:07.365549 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-05b47212-a38f-4c8d-8ae9-c670e484e976 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:07.367312 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-05b47212-a38f-4c8d-8ae9-c670e484e976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:07.375275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1380/2795] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:26:07.379434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b47212-a38f-4c8d-8ae9-c670e484e976 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:07.389013 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b47212-a38f-4c8d-8ae9-c670e484e976 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:07.394000 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56e6d8-eb9b-4403-97e7-63df2af73657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:07.394591 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56e6d8-eb9b-4403-97e7-63df2af73657 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:07.394870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56e6d8-eb9b-4403-97e7-63df2af73657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:07.395177 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56e6d8-eb9b-4403-97e7-63df2af73657 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:07.395812 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56e6d8-eb9b-4403-97e7-63df2af73657 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:26:07.396141 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56e6d8-eb9b-4403-97e7-63df2af73657 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:07.397785 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56e6d8-eb9b-4403-97e7-63df2af73657 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:07.412451 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1381/2796] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 08:26:07.425456 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56e6d8-eb9b-4403-97e7-63df2af73657 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:07.436448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56e6d8-eb9b-4403-97e7-63df2af73657 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:07.441244 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f4e4d-461f-494b-a0bc-ee53d2a5afa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:07.441847 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f4e4d-461f-494b-a0bc-ee53d2a5afa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:07.442129 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f4e4d-461f-494b-a0bc-ee53d2a5afa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:07.443032 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f4e4d-461f-494b-a0bc-ee53d2a5afa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:07.443129 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f4e4d-461f-494b-a0bc-ee53d2a5afa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:07.445743 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f4e4d-461f-494b-a0bc-ee53d2a5afa9 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:07.552825 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1382/2797] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:26:07 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:26:07.558171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f4e4d-461f-494b-a0bc-ee53d2a5afa9 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:07.562446 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-af0011d3-e563-43f1-8cb2-2195b28357da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97b2b03113c7495d996dbe062bb3bfde'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:07.583480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-af0011d3-e563-43f1-8cb2-2195b28357da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97b2b03113c7495d996dbe062bb3bfde'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:07.584310 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-af0011d3-e563-43f1-8cb2-2195b28357da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97b2b03113c7495d996dbe062bb3bfde'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:07.636261 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f4e4d-461f-494b-a0bc-ee53d2a5afa9 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:07.641287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac1d97-9525-4a08-b9c8-3ca1f5bc334d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:07.641893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac1d97-9525-4a08-b9c8-3ca1f5bc334d admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:07.642168 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac1d97-9525-4a08-b9c8-3ca1f5bc334d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:07.642490 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac1d97-9525-4a08-b9c8-3ca1f5bc334d admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:07.643150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac1d97-9525-4a08-b9c8-3ca1f5bc334d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:07.645079 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac1d97-9525-4a08-b9c8-3ca1f5bc334d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:07.656182 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-af0011d3-e563-43f1-8cb2-2195b28357da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97b2b03113c7495d996dbe062bb3bfde'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:07.657599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-af0011d3-e563-43f1-8cb2-2195b28357da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97b2b03113c7495d996dbe062bb3bfde'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:07.673060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1416/2798] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:26:07 2022] DELETE /identity/v3/users/97b2b03113c7495d996dbe062bb3bfde => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:26:07.677693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42506b-5218-4189-abff-56b57bf68deb None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:07.678042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42506b-5218-4189-abff-56b57bf68deb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:07.678412 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42506b-5218-4189-abff-56b57bf68deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:07.699699 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8c42506b-5218-4189-abff-56b57bf68deb None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:07.701015 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8c42506b-5218-4189-abff-56b57bf68deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:07.728176 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:07.753893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1383/2799] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:26:07 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:26:07.758086 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac1d97-9525-4a08-b9c8-3ca1f5bc334d admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:07.759892 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1fac1d97-9525-4a08-b9c8-3ca1f5bc334d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:07.770117 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1417/2800] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:26:07.773867 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac1d97-9525-4a08-b9c8-3ca1f5bc334d admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:07.780000 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89641df1-53c1-4d31-8a44-3c735dfac095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:07.780778 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89641df1-53c1-4d31-8a44-3c735dfac095 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:07.781093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42506b-5218-4189-abff-56b57bf68deb None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:07.781209 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89641df1-53c1-4d31-8a44-3c735dfac095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:07.781373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89641df1-53c1-4d31-8a44-3c735dfac095 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:07.782265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89641df1-53c1-4d31-8a44-3c735dfac095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:07.783164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8c42506b-5218-4189-abff-56b57bf68deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:07.786482 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89641df1-53c1-4d31-8a44-3c735dfac095 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:07.795987 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:07.802516 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1384/2801] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:26:07.807281 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89641df1-53c1-4d31-8a44-3c735dfac095 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:07.825579 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89641df1-53c1-4d31-8a44-3c735dfac095 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:07.833936 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea27e36-141a-4cc7-b5de-2620c6b62449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:07.834604 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea27e36-141a-4cc7-b5de-2620c6b62449 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:07.834880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea27e36-141a-4cc7-b5de-2620c6b62449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:07.836338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea27e36-141a-4cc7-b5de-2620c6b62449 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:07.836748 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea27e36-141a-4cc7-b5de-2620c6b62449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:07.838366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea27e36-141a-4cc7-b5de-2620c6b62449 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:07.851980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1385/2802] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:26:07.863197 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea27e36-141a-4cc7-b5de-2620c6b62449 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:07.873677 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea27e36-141a-4cc7-b5de-2620c6b62449 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:07.874664 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42506b-5218-4189-abff-56b57bf68deb None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:07.878698 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe9fe2-018e-4ea8-8ffe-6e4c750b682b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:07.879398 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe9fe2-018e-4ea8-8ffe-6e4c750b682b admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:07.879605 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe258963-db0e-4e6f-b3d1-b7a6fdf377a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:07.879745 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe9fe2-018e-4ea8-8ffe-6e4c750b682b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:07.879948 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe9fe2-018e-4ea8-8ffe-6e4c750b682b admin admin] PATH_INFO: `/v3/projects/2156f5b3f22d45bc9ce6188fb962d28f/users/76bf55964c804df988602317e8ea4169/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:07.880190 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe258963-db0e-4e6f-b3d1-b7a6fdf377a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:07.881131 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe258963-db0e-4e6f-b3d1-b7a6fdf377a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:07.882128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe258963-db0e-4e6f-b3d1-b7a6fdf377a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:07.893088 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe258963-db0e-4e6f-b3d1-b7a6fdf377a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:07.894626 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe258963-db0e-4e6f-b3d1-b7a6fdf377a5 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:07.898126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe9fe2-018e-4ea8-8ffe-6e4c750b682b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2156f5b3f22d45bc9ce6188fb962d28f, user_id=76bf55964c804df988602317e8ea4169, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:07.901562 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe9fe2-018e-4ea8-8ffe-6e4c750b682b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:07.906183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1418/2803] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:26:07.986013 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1386/2804] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:26:07 2022] PUT /identity/v3/projects/2156f5b3f22d45bc9ce6188fb962d28f/users/76bf55964c804df988602317e8ea4169/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:26:07.998113 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4632f4b-a3ee-454e-91d4-f966cf81e18e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:07.998397 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4632f4b-a3ee-454e-91d4-f966cf81e18e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:07.998675 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4632f4b-a3ee-454e-91d4-f966cf81e18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:08.032341 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a4632f4b-a3ee-454e-91d4-f966cf81e18e None None] MFA Rules not processed for user `76bf55964c804df988602317e8ea4169`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:08.033718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a4632f4b-a3ee-454e-91d4-f966cf81e18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:08.109819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1419/2805] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:08.118790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe9fe2-018e-4ea8-8ffe-6e4c750b682b admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:08.121072 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-74fe9fe2-018e-4ea8-8ffe-6e4c750b682b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:08.217203 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe9fe2-018e-4ea8-8ffe-6e4c750b682b admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:08.227112 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34bd0c-2777-4750-9d78-89d25119ad61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:08.228093 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34bd0c-2777-4750-9d78-89d25119ad61 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:08.229352 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34bd0c-2777-4750-9d78-89d25119ad61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:08.229930 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34bd0c-2777-4750-9d78-89d25119ad61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:08.257997 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34bd0c-2777-4750-9d78-89d25119ad61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:08.260014 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34bd0c-2777-4750-9d78-89d25119ad61 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:08.272926 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1387/2806] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:26:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:26:08.469782 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4632f4b-a3ee-454e-91d4-f966cf81e18e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:08.471286 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a4632f4b-a3ee-454e-91d4-f966cf81e18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:08.550544 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4632f4b-a3ee-454e-91d4-f966cf81e18e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:08.556092 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1c178-6e1d-4053-ac74-7dba99d8f563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:08.556728 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1c178-6e1d-4053-ac74-7dba99d8f563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:08.557356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1c178-6e1d-4053-ac74-7dba99d8f563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:08.557842 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1c178-6e1d-4053-ac74-7dba99d8f563 admin admin] PATH_INFO: `/v3/projects/96d123f1bcdb4a10a756f01a2e1d5746` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:08.559673 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1c178-6e1d-4053-ac74-7dba99d8f563 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96d123f1bcdb4a10a756f01a2e1d5746)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:08.562532 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1c178-6e1d-4053-ac74-7dba99d8f563 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:08.570414 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:08.729411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4bc1c178-6e1d-4053-ac74-7dba99d8f563 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96d123f1bcdb4a10a756f01a2e1d5746', 'request_id': 'req-4bc1c178-6e1d-4053-ac74-7dba99d8f563'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:08.735577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4bc1c178-6e1d-4053-ac74-7dba99d8f563 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96d123f1bcdb4a10a756f01a2e1d5746', 'request_id': 'req-4bc1c178-6e1d-4053-ac74-7dba99d8f563'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:08.849442 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4bc1c178-6e1d-4053-ac74-7dba99d8f563 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96d123f1bcdb4a10a756f01a2e1d5746', 'request_id': 'req-4bc1c178-6e1d-4053-ac74-7dba99d8f563'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:08.861748 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4bc1c178-6e1d-4053-ac74-7dba99d8f563 admin admin] The token cache is being invalidate because project 96d123f1bcdb4a10a756f01a2e1d5746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:26:08.868935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4bc1c178-6e1d-4053-ac74-7dba99d8f563 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:08.868935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4bc1c178-6e1d-4053-ac74-7dba99d8f563 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:08.868935 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1420/2807] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:26:08 2022] DELETE /identity/v3/projects/96d123f1bcdb4a10a756f01a2e1d5746 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:08.869217 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34bd0c-2777-4750-9d78-89d25119ad61 service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:08.879649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34bd0c-2777-4750-9d78-89d25119ad61 service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:08.940033 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ecaf4-b022-4185-8f48-2cb580e092e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:08.940972 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ecaf4-b022-4185-8f48-2cb580e092e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:08.941248 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ecaf4-b022-4185-8f48-2cb580e092e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:08.941520 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ecaf4-b022-4185-8f48-2cb580e092e1 admin admin] PATH_INFO: `/v3/users/c74f13a0885a4a4685c45e821987b301` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:08.942642 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ecaf4-b022-4185-8f48-2cb580e092e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c74f13a0885a4a4685c45e821987b301)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:08.945171 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ecaf4-b022-4185-8f48-2cb580e092e1 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:09.320118 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-2b6ecaf4-b022-4185-8f48-2cb580e092e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c74f13a0885a4a4685c45e821987b301'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:09.320798 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-2b6ecaf4-b022-4185-8f48-2cb580e092e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c74f13a0885a4a4685c45e821987b301'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:09.339929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-2b6ecaf4-b022-4185-8f48-2cb580e092e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c74f13a0885a4a4685c45e821987b301'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:09.352355 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:09.454243 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-2b6ecaf4-b022-4185-8f48-2cb580e092e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c74f13a0885a4a4685c45e821987b301'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:09.454954 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-2b6ecaf4-b022-4185-8f48-2cb580e092e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c74f13a0885a4a4685c45e821987b301'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:09.479781 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1388/2808] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:26:08 2022] DELETE /identity/v3/users/c74f13a0885a4a4685c45e821987b301 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:09.920732 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1c178-6e1d-4053-ac74-7dba99d8f563 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:09.922216 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4bc1c178-6e1d-4053-ac74-7dba99d8f563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:09.944285 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1c178-6e1d-4053-ac74-7dba99d8f563 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:10.012413 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c744218-14db-4166-bf5b-253d34747010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:10.013039 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c744218-14db-4166-bf5b-253d34747010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:10.013368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c744218-14db-4166-bf5b-253d34747010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:10.013642 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c744218-14db-4166-bf5b-253d34747010 admin admin] PATH_INFO: `/v3/projects/5324f53294754decb9e192a6f9ae2950` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:10.014839 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c744218-14db-4166-bf5b-253d34747010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5324f53294754decb9e192a6f9ae2950)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:10.017075 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c744218-14db-4166-bf5b-253d34747010 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:10.122523 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0c744218-14db-4166-bf5b-253d34747010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5324f53294754decb9e192a6f9ae2950', 'request_id': 'req-0c744218-14db-4166-bf5b-253d34747010'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:10.128777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0c744218-14db-4166-bf5b-253d34747010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5324f53294754decb9e192a6f9ae2950', 'request_id': 'req-0c744218-14db-4166-bf5b-253d34747010'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:10.243468 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0c744218-14db-4166-bf5b-253d34747010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5324f53294754decb9e192a6f9ae2950', 'request_id': 'req-0c744218-14db-4166-bf5b-253d34747010'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:10.275551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0c744218-14db-4166-bf5b-253d34747010 admin admin] The token cache is being invalidate because project 5324f53294754decb9e192a6f9ae2950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:26:10.275819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0c744218-14db-4166-bf5b-253d34747010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:10.276672 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0c744218-14db-4166-bf5b-253d34747010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:10.283636 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1421/2809] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:26:09 2022] DELETE /identity/v3/projects/5324f53294754decb9e192a6f9ae2950 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:10.285913 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742685a5-889d-422c-b4f8-d4d11f5f57d4 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:10.286640 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742685a5-889d-422c-b4f8-d4d11f5f57d4 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:10.287049 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742685a5-889d-422c-b4f8-d4d11f5f57d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:10.325399 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-742685a5-889d-422c-b4f8-d4d11f5f57d4 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:10.326929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-742685a5-889d-422c-b4f8-d4d11f5f57d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:10.407891 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1389/2810] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:10.416751 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca477897-4149-4933-b162-22f97a01722a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:10.417193 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca477897-4149-4933-b162-22f97a01722a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:10.417688 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca477897-4149-4933-b162-22f97a01722a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:10.440872 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ca477897-4149-4933-b162-22f97a01722a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:10.442109 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ca477897-4149-4933-b162-22f97a01722a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:10.451391 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1422/2811] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:10.455716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742685a5-889d-422c-b4f8-d4d11f5f57d4 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:10.465844 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742685a5-889d-422c-b4f8-d4d11f5f57d4 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:10.470535 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70066814-a420-4d2a-b960-e191c2feebd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:10.471148 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70066814-a420-4d2a-b960-e191c2feebd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:10.471421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70066814-a420-4d2a-b960-e191c2feebd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:10.471686 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70066814-a420-4d2a-b960-e191c2feebd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:10.472323 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70066814-a420-4d2a-b960-e191c2feebd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:26:10.472618 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70066814-a420-4d2a-b960-e191c2feebd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:10.474206 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70066814-a420-4d2a-b960-e191c2feebd5 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:10.486422 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1390/2812] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:26:10.490756 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca477897-4149-4933-b162-22f97a01722a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:10.502879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca477897-4149-4933-b162-22f97a01722a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:10.508923 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828b50e-a78e-4049-b8a3-42e308e4089e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:10.509540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828b50e-a78e-4049-b8a3-42e308e4089e admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:10.509833 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828b50e-a78e-4049-b8a3-42e308e4089e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:10.510097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828b50e-a78e-4049-b8a3-42e308e4089e admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:10.510738 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828b50e-a78e-4049-b8a3-42e308e4089e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:10.515829 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828b50e-a78e-4049-b8a3-42e308e4089e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:10.536307 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:10.644077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1423/2813] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:26:10 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:26:10.648481 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70066814-a420-4d2a-b960-e191c2feebd5 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:10.658415 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70066814-a420-4d2a-b960-e191c2feebd5 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:10.714585 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313166e1-f075-4c45-91f6-d4a6907de734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:10.715214 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313166e1-f075-4c45-91f6-d4a6907de734 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:10.715486 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313166e1-f075-4c45-91f6-d4a6907de734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:10.720023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313166e1-f075-4c45-91f6-d4a6907de734 admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:10.720023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313166e1-f075-4c45-91f6-d4a6907de734 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:10.720023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313166e1-f075-4c45-91f6-d4a6907de734 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:10.840423 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1391/2814] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:26:10 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:26:10.844939 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828b50e-a78e-4049-b8a3-42e308e4089e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:10.856990 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828b50e-a78e-4049-b8a3-42e308e4089e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:10.861967 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2b0ed-218a-4b77-9ff4-999d18de5673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:10.862581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2b0ed-218a-4b77-9ff4-999d18de5673 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:10.862860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2b0ed-218a-4b77-9ff4-999d18de5673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:10.863170 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2b0ed-218a-4b77-9ff4-999d18de5673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:10.863859 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2b0ed-218a-4b77-9ff4-999d18de5673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:10.865568 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2b0ed-218a-4b77-9ff4-999d18de5673 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:10.876184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1424/2815] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:26:10.880349 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313166e1-f075-4c45-91f6-d4a6907de734 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:10.891311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313166e1-f075-4c45-91f6-d4a6907de734 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:10.896557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497f7c5-7750-4a09-a98a-0f0e25ed1d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:10.897155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497f7c5-7750-4a09-a98a-0f0e25ed1d20 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:10.897432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497f7c5-7750-4a09-a98a-0f0e25ed1d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:10.898277 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497f7c5-7750-4a09-a98a-0f0e25ed1d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:10.898866 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497f7c5-7750-4a09-a98a-0f0e25ed1d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:10.900498 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497f7c5-7750-4a09-a98a-0f0e25ed1d20 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:10.911535 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1392/2816] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:26:10.916581 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2b0ed-218a-4b77-9ff4-999d18de5673 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:10.927983 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2b0ed-218a-4b77-9ff4-999d18de5673 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:10.934730 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2eb744-d9c4-4b54-aff6-00b1d12c595e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:10.935449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2eb744-d9c4-4b54-aff6-00b1d12c595e admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:10.935727 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2eb744-d9c4-4b54-aff6-00b1d12c595e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:10.936025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2eb744-d9c4-4b54-aff6-00b1d12c595e admin admin] PATH_INFO: `/v3/projects/6814ab60f34c4f9ca94304b970f35073/users/5145dd0fd2f84906a8d7a18ff59ff55f/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:10.952426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2eb744-d9c4-4b54-aff6-00b1d12c595e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6814ab60f34c4f9ca94304b970f35073, user_id=5145dd0fd2f84906a8d7a18ff59ff55f, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:10.955213 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2eb744-d9c4-4b54-aff6-00b1d12c595e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:11.033512 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1425/2817] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:26:10 2022] PUT /identity/v3/projects/6814ab60f34c4f9ca94304b970f35073/users/5145dd0fd2f84906a8d7a18ff59ff55f/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:11.044711 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31ff29-6b95-4d50-8500-31a44ed168e8 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:11.046637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31ff29-6b95-4d50-8500-31a44ed168e8 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:11.046637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31ff29-6b95-4d50-8500-31a44ed168e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:11.075026 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-4e31ff29-6b95-4d50-8500-31a44ed168e8 None None] MFA Rules not processed for user `5145dd0fd2f84906a8d7a18ff59ff55f`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:11.076284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4e31ff29-6b95-4d50-8500-31a44ed168e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:11.120546 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:11.143678 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1393/2818] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:11.155602 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2eb744-d9c4-4b54-aff6-00b1d12c595e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:11.157291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3c2eb744-d9c4-4b54-aff6-00b1d12c595e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:11.238374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2eb744-d9c4-4b54-aff6-00b1d12c595e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:11.242368 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089abd0-c26c-4e14-8d0d-b0d36c50e893 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30909d152a5c4a1fac6c1a2765868177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:11.242992 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089abd0-c26c-4e14-8d0d-b0d36c50e893 service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:11.243275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089abd0-c26c-4e14-8d0d-b0d36c50e893 service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:11.243548 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089abd0-c26c-4e14-8d0d-b0d36c50e893 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:11.251663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089abd0-c26c-4e14-8d0d-b0d36c50e893 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:11.253093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089abd0-c26c-4e14-8d0d-b0d36c50e893 service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:11.262879 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1426/2819] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:26:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:26:13.838833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31ff29-6b95-4d50-8500-31a44ed168e8 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:13.840287 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-4e31ff29-6b95-4d50-8500-31a44ed168e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:13.907587 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31ff29-6b95-4d50-8500-31a44ed168e8 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:13.912443 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8e7f3-6177-48e8-a21e-a94065101820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:13.913126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8e7f3-6177-48e8-a21e-a94065101820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:13.913437 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8e7f3-6177-48e8-a21e-a94065101820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:13.913665 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8e7f3-6177-48e8-a21e-a94065101820 admin admin] PATH_INFO: `/v3/users/76bf55964c804df988602317e8ea4169` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:13.914823 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8e7f3-6177-48e8-a21e-a94065101820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76bf55964c804df988602317e8ea4169)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:13.916807 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8e7f3-6177-48e8-a21e-a94065101820 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:14.147473 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a3f8e7f3-6177-48e8-a21e-a94065101820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76bf55964c804df988602317e8ea4169'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:14.148760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a3f8e7f3-6177-48e8-a21e-a94065101820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76bf55964c804df988602317e8ea4169'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:14.172840 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a3f8e7f3-6177-48e8-a21e-a94065101820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76bf55964c804df988602317e8ea4169'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:14.243058 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a3f8e7f3-6177-48e8-a21e-a94065101820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76bf55964c804df988602317e8ea4169'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:14.243746 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-a3f8e7f3-6177-48e8-a21e-a94065101820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76bf55964c804df988602317e8ea4169'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:14.256787 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1394/2820] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:26:13 2022] DELETE /identity/v3/users/76bf55964c804df988602317e8ea4169 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:14.261396 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd429e4-6c16-4f58-b46e-bc1c5b0bc8de None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:14.261877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd429e4-6c16-4f58-b46e-bc1c5b0bc8de None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:14.261979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd429e4-6c16-4f58-b46e-bc1c5b0bc8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:14.284841 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0cd429e4-6c16-4f58-b46e-bc1c5b0bc8de None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:14.286131 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0cd429e4-6c16-4f58-b46e-bc1c5b0bc8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:14.347629 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1427/2821] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:14.358388 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8e7f3-6177-48e8-a21e-a94065101820 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:14.359903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a3f8e7f3-6177-48e8-a21e-a94065101820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:14.419832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8e7f3-6177-48e8-a21e-a94065101820 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:14.423362 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144801b-bbbf-41f1-8521-9098251412fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:14.423978 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144801b-bbbf-41f1-8521-9098251412fa service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:14.424254 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144801b-bbbf-41f1-8521-9098251412fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:14.424525 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144801b-bbbf-41f1-8521-9098251412fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:14.434327 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5144801b-bbbf-41f1-8521-9098251412fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:14.435814 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5144801b-bbbf-41f1-8521-9098251412fa service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:14.445139 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1395/2822] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:26:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:26:14.910320 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd429e4-6c16-4f58-b46e-bc1c5b0bc8de None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:14.910900 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0cd429e4-6c16-4f58-b46e-bc1c5b0bc8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:14.925337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd429e4-6c16-4f58-b46e-bc1c5b0bc8de None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:14.930252 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4d9ca-2f71-4e0e-916e-1bb9374527ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:14.930889 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4d9ca-2f71-4e0e-916e-1bb9374527ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:14.933285 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4d9ca-2f71-4e0e-916e-1bb9374527ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:14.933285 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4d9ca-2f71-4e0e-916e-1bb9374527ef admin admin] PATH_INFO: `/v3/projects/2156f5b3f22d45bc9ce6188fb962d28f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:14.933285 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a4d9ca-2f71-4e0e-916e-1bb9374527ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=2156f5b3f22d45bc9ce6188fb962d28f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:14.935042 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a4d9ca-2f71-4e0e-916e-1bb9374527ef admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:15.073706 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:15.080536 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-78a4d9ca-2f71-4e0e-916e-1bb9374527ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2156f5b3f22d45bc9ce6188fb962d28f', 'request_id': 'req-78a4d9ca-2f71-4e0e-916e-1bb9374527ef'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:15.087530 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-78a4d9ca-2f71-4e0e-916e-1bb9374527ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2156f5b3f22d45bc9ce6188fb962d28f', 'request_id': 'req-78a4d9ca-2f71-4e0e-916e-1bb9374527ef'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:15.169735 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-78a4d9ca-2f71-4e0e-916e-1bb9374527ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2156f5b3f22d45bc9ce6188fb962d28f', 'request_id': 'req-78a4d9ca-2f71-4e0e-916e-1bb9374527ef'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:15.181141 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-78a4d9ca-2f71-4e0e-916e-1bb9374527ef admin admin] The token cache is being invalidate because project 2156f5b3f22d45bc9ce6188fb962d28f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:26:15.181629 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-78a4d9ca-2f71-4e0e-916e-1bb9374527ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:15.182614 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-78a4d9ca-2f71-4e0e-916e-1bb9374527ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:15.184133 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1428/2823] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:26:14 2022] DELETE /identity/v3/projects/2156f5b3f22d45bc9ce6188fb962d28f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:15.188990 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec4473-18ba-4b9b-a297-f42db46493f7 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:15.189515 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec4473-18ba-4b9b-a297-f42db46493f7 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:15.189515 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec4473-18ba-4b9b-a297-f42db46493f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:15.199268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:15.212830 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-e3ec4473-18ba-4b9b-a297-f42db46493f7 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:15.218430 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-e3ec4473-18ba-4b9b-a297-f42db46493f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:15.293657 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1396/2824] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:15.301623 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62df0a0-67a1-4566-8801-4f084098d005 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:15.302096 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62df0a0-67a1-4566-8801-4f084098d005 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:15.302195 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62df0a0-67a1-4566-8801-4f084098d005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:15.323126 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b62df0a0-67a1-4566-8801-4f084098d005 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:15.324246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b62df0a0-67a1-4566-8801-4f084098d005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:15.332462 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1429/2825] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:15.337383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec4473-18ba-4b9b-a297-f42db46493f7 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:15.346962 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec4473-18ba-4b9b-a297-f42db46493f7 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:15.351649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb80d5-0b58-4d1c-a06d-4a1d224759ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:15.352237 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb80d5-0b58-4d1c-a06d-4a1d224759ca admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:15.352513 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb80d5-0b58-4d1c-a06d-4a1d224759ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:15.352782 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb80d5-0b58-4d1c-a06d-4a1d224759ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:15.353421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb80d5-0b58-4d1c-a06d-4a1d224759ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:26:15.353722 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb80d5-0b58-4d1c-a06d-4a1d224759ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:15.355467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb80d5-0b58-4d1c-a06d-4a1d224759ca admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:15.368126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1397/2826] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:26:15.372616 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62df0a0-67a1-4566-8801-4f084098d005 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:15.382996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62df0a0-67a1-4566-8801-4f084098d005 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:15.387837 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e12925-551f-42d3-bd42-d64f3d64a5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:15.388449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e12925-551f-42d3-bd42-d64f3d64a5eb admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:15.388725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e12925-551f-42d3-bd42-d64f3d64a5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:15.388994 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e12925-551f-42d3-bd42-d64f3d64a5eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:15.389635 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e12925-551f-42d3-bd42-d64f3d64a5eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:15.391979 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e12925-551f-42d3-bd42-d64f3d64a5eb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:15.499411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1430/2827] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:26:15 2022] POST /identity/v3/projects => generated 369 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:26:15.503739 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb80d5-0b58-4d1c-a06d-4a1d224759ca admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:15.514669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb80d5-0b58-4d1c-a06d-4a1d224759ca admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:15.571238 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5ad68-2ff0-488d-a605-8150b2e76eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:15.571834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5ad68-2ff0-488d-a605-8150b2e76eec admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:15.572108 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5ad68-2ff0-488d-a605-8150b2e76eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:15.572375 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5ad68-2ff0-488d-a605-8150b2e76eec admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:15.572993 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5ad68-2ff0-488d-a605-8150b2e76eec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:15.574893 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5ad68-2ff0-488d-a605-8150b2e76eec admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:15.673166 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1398/2828] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:26:15 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:26:15.679940 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e12925-551f-42d3-bd42-d64f3d64a5eb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:15.690230 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e12925-551f-42d3-bd42-d64f3d64a5eb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:15.695498 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202c1d3-7712-44ea-af8d-4fe581b5685b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:15.696076 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202c1d3-7712-44ea-af8d-4fe581b5685b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:15.696346 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202c1d3-7712-44ea-af8d-4fe581b5685b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:15.696615 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202c1d3-7712-44ea-af8d-4fe581b5685b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:15.697275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202c1d3-7712-44ea-af8d-4fe581b5685b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:15.698857 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202c1d3-7712-44ea-af8d-4fe581b5685b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:15.708965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1431/2829] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:26:15.712955 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5ad68-2ff0-488d-a605-8150b2e76eec admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:15.723176 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5ad68-2ff0-488d-a605-8150b2e76eec admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:15.728387 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e7d3c-264f-43f3-8fb6-eefd4eac8c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:15.728977 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e7d3c-264f-43f3-8fb6-eefd4eac8c7f admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:15.729253 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e7d3c-264f-43f3-8fb6-eefd4eac8c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:15.729569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e7d3c-264f-43f3-8fb6-eefd4eac8c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:15.730191 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e7d3c-264f-43f3-8fb6-eefd4eac8c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:15.731888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e7d3c-264f-43f3-8fb6-eefd4eac8c7f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:15.745917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1399/2830] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:26:15.750165 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202c1d3-7712-44ea-af8d-4fe581b5685b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:15.760127 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202c1d3-7712-44ea-af8d-4fe581b5685b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:15.764773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46408007-e4f2-4741-9fea-350c39c3dd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:15.765669 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46408007-e4f2-4741-9fea-350c39c3dd92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:15.765943 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46408007-e4f2-4741-9fea-350c39c3dd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:15.766246 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46408007-e4f2-4741-9fea-350c39c3dd92 admin admin] PATH_INFO: `/v3/projects/3e20d4ae8a634548bfd9540de8ab5653/users/a1823fec359f4b1c8b2ad6360b1bc6e3/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:15.780612 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46408007-e4f2-4741-9fea-350c39c3dd92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e20d4ae8a634548bfd9540de8ab5653, user_id=a1823fec359f4b1c8b2ad6360b1bc6e3, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:15.783289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46408007-e4f2-4741-9fea-350c39c3dd92 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:15.852083 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1432/2831] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:26:15 2022] PUT /identity/v3/projects/3e20d4ae8a634548bfd9540de8ab5653/users/a1823fec359f4b1c8b2ad6360b1bc6e3/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:15.862799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd8d10-baa8-4de2-a2ce-ec3f673b1599 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:15.863107 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd8d10-baa8-4de2-a2ce-ec3f673b1599 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:15.863392 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd8d10-baa8-4de2-a2ce-ec3f673b1599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:15.891115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-5cdd8d10-baa8-4de2-a2ce-ec3f673b1599 None None] MFA Rules not processed for user `a1823fec359f4b1c8b2ad6360b1bc6e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:15.892340 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5cdd8d10-baa8-4de2-a2ce-ec3f673b1599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:15.958147 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:15.962603 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1400/2832] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:24.672326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46408007-e4f2-4741-9fea-350c39c3dd92 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:24.674062 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-46408007-e4f2-4741-9fea-350c39c3dd92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:24.731749 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46408007-e4f2-4741-9fea-350c39c3dd92 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:24.737587 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b90f7a-b43f-48ac-a274-9d6416a3227b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30909d152a5c4a1fac6c1a2765868177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:24.738254 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b90f7a-b43f-48ac-a274-9d6416a3227b service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:24.738459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b90f7a-b43f-48ac-a274-9d6416a3227b service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:24.738896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b90f7a-b43f-48ac-a274-9d6416a3227b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:24.748263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b90f7a-b43f-48ac-a274-9d6416a3227b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:24.749693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b90f7a-b43f-48ac-a274-9d6416a3227b service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:24.758863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1433/2833] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:26:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:26:34.754511 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=63,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:45.443701 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc341e1-106d-4988-a291-01ebcd574d78 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:45.444216 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc341e1-106d-4988-a291-01ebcd574d78 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:45.444216 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc341e1-106d-4988-a291-01ebcd574d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:45.463990 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-7bc341e1-106d-4988-a291-01ebcd574d78 None None] MFA Rules not processed for user `79d29aa9482f4c1f9b1554333d85d80a`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:45.465205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7bc341e1-106d-4988-a291-01ebcd574d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:45.520109 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1401/2834] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:26:46.644497 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b90f7a-b43f-48ac-a274-9d6416a3227b service glance] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:46.645936 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c4b90f7a-b43f-48ac-a274-9d6416a3227b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:46.718461 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b90f7a-b43f-48ac-a274-9d6416a3227b service glance] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:46.723500 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055b85e-9dbe-4942-af67-e0be79973da5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6074872303d3468fb566378fa34e8bea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:46.724103 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055b85e-9dbe-4942-af67-e0be79973da5 service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:46.724378 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055b85e-9dbe-4942-af67-e0be79973da5 service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:46.724644 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055b85e-9dbe-4942-af67-e0be79973da5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:46.726365 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9055b85e-9dbe-4942-af67-e0be79973da5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:46.738263 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055b85e-9dbe-4942-af67-e0be79973da5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:46.739885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055b85e-9dbe-4942-af67-e0be79973da5 service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:46.749768 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1434/2835] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:26:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:26:50.733880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc341e1-106d-4988-a291-01ebcd574d78 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:50.736189 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7bc341e1-106d-4988-a291-01ebcd574d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:50.817216 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc341e1-106d-4988-a291-01ebcd574d78 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:50.822038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fed85e-bb6c-451d-86ef-326134e4e79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:50.822758 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fed85e-bb6c-451d-86ef-326134e4e79e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:50.823162 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fed85e-bb6c-451d-86ef-326134e4e79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:50.823430 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fed85e-bb6c-451d-86ef-326134e4e79e admin admin] PATH_INFO: `/v3/users/5145dd0fd2f84906a8d7a18ff59ff55f` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:50.824556 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fed85e-bb6c-451d-86ef-326134e4e79e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5145dd0fd2f84906a8d7a18ff59ff55f)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:50.827167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fed85e-bb6c-451d-86ef-326134e4e79e admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:51.046225 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d8fed85e-bb6c-451d-86ef-326134e4e79e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5145dd0fd2f84906a8d7a18ff59ff55f'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:51.046889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d8fed85e-bb6c-451d-86ef-326134e4e79e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5145dd0fd2f84906a8d7a18ff59ff55f'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:51.058797 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d8fed85e-bb6c-451d-86ef-326134e4e79e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5145dd0fd2f84906a8d7a18ff59ff55f'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:51.190268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d8fed85e-bb6c-451d-86ef-326134e4e79e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5145dd0fd2f84906a8d7a18ff59ff55f'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:51.190964 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d8fed85e-bb6c-451d-86ef-326134e4e79e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5145dd0fd2f84906a8d7a18ff59ff55f'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:51.206324 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1402/2836] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:26:50 2022] DELETE /identity/v3/users/5145dd0fd2f84906a8d7a18ff59ff55f => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:51.211589 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d950938-2ad2-4318-8020-f062ae5c7ffc None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:51.212123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d950938-2ad2-4318-8020-f062ae5c7ffc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:51.212123 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d950938-2ad2-4318-8020-f062ae5c7ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:51.233742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3d950938-2ad2-4318-8020-f062ae5c7ffc None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:51.235250 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3d950938-2ad2-4318-8020-f062ae5c7ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:51.305034 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1435/2837] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:51.315852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fed85e-bb6c-451d-86ef-326134e4e79e admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:51.317323 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d8fed85e-bb6c-451d-86ef-326134e4e79e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:51.347685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:51.377378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fed85e-bb6c-451d-86ef-326134e4e79e admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:51.381062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4126cc2-2e88-4230-a2db-87075630826a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:51.381655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4126cc2-2e88-4230-a2db-87075630826a service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:51.383067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4126cc2-2e88-4230-a2db-87075630826a service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:51.383067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4126cc2-2e88-4230-a2db-87075630826a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:51.391568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4126cc2-2e88-4230-a2db-87075630826a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:51.393060 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4126cc2-2e88-4230-a2db-87075630826a service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:51.402832 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1403/2838] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:26:51.853085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d950938-2ad2-4318-8020-f062ae5c7ffc None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:51.853085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3d950938-2ad2-4318-8020-f062ae5c7ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:51.865794 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d950938-2ad2-4318-8020-f062ae5c7ffc None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:51.871162 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a21e41-7a97-43c9-8657-1b128a760567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:51.871858 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a21e41-7a97-43c9-8657-1b128a760567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:51.872287 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a21e41-7a97-43c9-8657-1b128a760567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:51.873102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a21e41-7a97-43c9-8657-1b128a760567 admin admin] PATH_INFO: `/v3/projects/6814ab60f34c4f9ca94304b970f35073` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:51.874321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a21e41-7a97-43c9-8657-1b128a760567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6814ab60f34c4f9ca94304b970f35073)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:51.877283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a21e41-7a97-43c9-8657-1b128a760567 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:52.017792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-89a21e41-7a97-43c9-8657-1b128a760567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6814ab60f34c4f9ca94304b970f35073', 'request_id': 'req-89a21e41-7a97-43c9-8657-1b128a760567'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:52.023955 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-89a21e41-7a97-43c9-8657-1b128a760567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6814ab60f34c4f9ca94304b970f35073', 'request_id': 'req-89a21e41-7a97-43c9-8657-1b128a760567'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:52.109506 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-89a21e41-7a97-43c9-8657-1b128a760567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6814ab60f34c4f9ca94304b970f35073', 'request_id': 'req-89a21e41-7a97-43c9-8657-1b128a760567'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:52.120398 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-89a21e41-7a97-43c9-8657-1b128a760567 admin admin] The token cache is being invalidate because project 6814ab60f34c4f9ca94304b970f35073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:26:52.120655 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-89a21e41-7a97-43c9-8657-1b128a760567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:52.121271 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-89a21e41-7a97-43c9-8657-1b128a760567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:52.123406 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1436/2839] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:26:51 2022] DELETE /identity/v3/projects/6814ab60f34c4f9ca94304b970f35073 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:52.128395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7fcde-9869-4184-8a3b-1e624b7465a7 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:52.129173 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7fcde-9869-4184-8a3b-1e624b7465a7 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:52.129360 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7fcde-9869-4184-8a3b-1e624b7465a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:52.154914 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-36d7fcde-9869-4184-8a3b-1e624b7465a7 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:52.156668 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-36d7fcde-9869-4184-8a3b-1e624b7465a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:52.220885 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1404/2840] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:52.286085 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e06ec-65d1-4eb8-bc79-b51ffc633a4b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:52.286612 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e06ec-65d1-4eb8-bc79-b51ffc633a4b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:52.286612 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e06ec-65d1-4eb8-bc79-b51ffc633a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:52.306779 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6e1e06ec-65d1-4eb8-bc79-b51ffc633a4b None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:52.308001 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6e1e06ec-65d1-4eb8-bc79-b51ffc633a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:52.320936 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1437/2841] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:52.321435 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7fcde-9869-4184-8a3b-1e624b7465a7 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:52.330923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7fcde-9869-4184-8a3b-1e624b7465a7 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:52.335489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cac58-a1a3-4adf-8d3f-13fe3ce3b4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:52.336084 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cac58-a1a3-4adf-8d3f-13fe3ce3b4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:52.336355 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cac58-a1a3-4adf-8d3f-13fe3ce3b4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:52.336620 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cac58-a1a3-4adf-8d3f-13fe3ce3b4b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:52.337255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548cac58-a1a3-4adf-8d3f-13fe3ce3b4b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81479) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:26:52.337554 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548cac58-a1a3-4adf-8d3f-13fe3ce3b4b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:52.339187 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548cac58-a1a3-4adf-8d3f-13fe3ce3b4b3 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:52.352639 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1405/2842] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 06 08:26:52.358999 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e06ec-65d1-4eb8-bc79-b51ffc633a4b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:52.368583 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e06ec-65d1-4eb8-bc79-b51ffc633a4b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:52.373625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbea7f-dc55-4737-8e41-017b2b86ba04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:52.374240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbea7f-dc55-4737-8e41-017b2b86ba04 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:52.374518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbea7f-dc55-4737-8e41-017b2b86ba04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:52.374785 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbea7f-dc55-4737-8e41-017b2b86ba04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:52.375452 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbea7f-dc55-4737-8e41-017b2b86ba04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:52.377415 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbea7f-dc55-4737-8e41-017b2b86ba04 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:52.382239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:52.485639 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1438/2843] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:26:52 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:26:52.489899 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cac58-a1a3-4adf-8d3f-13fe3ce3b4b3 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:52.499902 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cac58-a1a3-4adf-8d3f-13fe3ce3b4b3 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:52.555850 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940b5e4-53c1-4db7-935b-1f6ef96bed2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:52.556368 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940b5e4-53c1-4db7-935b-1f6ef96bed2f admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:52.556643 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940b5e4-53c1-4db7-935b-1f6ef96bed2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:52.556908 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940b5e4-53c1-4db7-935b-1f6ef96bed2f admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:52.557519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940b5e4-53c1-4db7-935b-1f6ef96bed2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:52.559469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940b5e4-53c1-4db7-935b-1f6ef96bed2f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:52.664075 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1406/2844] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:26:52 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:26:52.669018 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbea7f-dc55-4737-8e41-017b2b86ba04 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:52.680757 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbea7f-dc55-4737-8e41-017b2b86ba04 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:52.685792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c25ea7-fe85-4e50-ae69-3271c31de0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:52.686390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c25ea7-fe85-4e50-ae69-3271c31de0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:52.686665 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c25ea7-fe85-4e50-ae69-3271c31de0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:52.686933 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c25ea7-fe85-4e50-ae69-3271c31de0e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:52.687740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c25ea7-fe85-4e50-ae69-3271c31de0e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:52.689321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c25ea7-fe85-4e50-ae69-3271c31de0e1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:52.699213 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1439/2845] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:26:52.704241 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940b5e4-53c1-4db7-935b-1f6ef96bed2f admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:52.714053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940b5e4-53c1-4db7-935b-1f6ef96bed2f admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:52.719377 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f2317-84a1-4036-807c-43f45172bed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:52.721605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f2317-84a1-4036-807c-43f45172bed7 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:52.721605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f2317-84a1-4036-807c-43f45172bed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:52.721605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f2317-84a1-4036-807c-43f45172bed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:52.721605 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f2317-84a1-4036-807c-43f45172bed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:52.722686 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f2317-84a1-4036-807c-43f45172bed7 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:52.733163 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1407/2846] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:26:52.737212 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c25ea7-fe85-4e50-ae69-3271c31de0e1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:52.747029 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c25ea7-fe85-4e50-ae69-3271c31de0e1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:52.752969 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ff84e-c312-4098-b4c5-9c6728b7b78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:52.754025 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ff84e-c312-4098-b4c5-9c6728b7b78d admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:52.754357 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ff84e-c312-4098-b4c5-9c6728b7b78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:52.754631 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ff84e-c312-4098-b4c5-9c6728b7b78d admin admin] PATH_INFO: `/v3/projects/98d174ba6a534f29b933594a420781dd/users/0b5e03c3c60649778a8f68de20675015/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:52.775390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ff84e-c312-4098-b4c5-9c6728b7b78d admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d174ba6a534f29b933594a420781dd, user_id=0b5e03c3c60649778a8f68de20675015, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:52.778039 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ff84e-c312-4098-b4c5-9c6728b7b78d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:52.883874 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1440/2847] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:26:52 2022] PUT /identity/v3/projects/98d174ba6a534f29b933594a420781dd/users/0b5e03c3c60649778a8f68de20675015/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:52.894504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f84daf-0c34-4a1f-9846-b313ce610e57 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:52.894895 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f84daf-0c34-4a1f-9846-b313ce610e57 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:52.895186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f84daf-0c34-4a1f-9846-b313ce610e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:52.923916 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-13f84daf-0c34-4a1f-9846-b313ce610e57 None None] MFA Rules not processed for user `0b5e03c3c60649778a8f68de20675015`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:52.925134 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-13f84daf-0c34-4a1f-9846-b313ce610e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:52.947172 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:52.996951 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1408/2848] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:53.001427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ff84e-c312-4098-b4c5-9c6728b7b78d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:53.063689 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ff84e-c312-4098-b4c5-9c6728b7b78d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:53.068372 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d817b-1b29-4385-8273-75ac01b446f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:53.068959 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d817b-1b29-4385-8273-75ac01b446f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:53.069235 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d817b-1b29-4385-8273-75ac01b446f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:53.069505 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d817b-1b29-4385-8273-75ac01b446f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:53.070139 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d817b-1b29-4385-8273-75ac01b446f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:53.072082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d817b-1b29-4385-8273-75ac01b446f6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:53.194403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1441/2849] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:26:53 2022] POST /identity/v3/projects => generated 375 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:26:53.198493 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f84daf-0c34-4a1f-9846-b313ce610e57 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:53.209792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f84daf-0c34-4a1f-9846-b313ce610e57 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:53.265540 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371eeb1-fdf4-47e6-b943-774c9b0295ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:53.269325 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371eeb1-fdf4-47e6-b943-774c9b0295ce admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:53.269325 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371eeb1-fdf4-47e6-b943-774c9b0295ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:53.269325 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371eeb1-fdf4-47e6-b943-774c9b0295ce admin admin] PATH_INFO: `/v3/users` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:53.269325 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371eeb1-fdf4-47e6-b943-774c9b0295ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:53.269325 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371eeb1-fdf4-47e6-b943-774c9b0295ce admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:53.358044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1409/2850] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:26:53 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:26:53.362004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d817b-1b29-4385-8273-75ac01b446f6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:53.372324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d817b-1b29-4385-8273-75ac01b446f6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:53.377560 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f10fbf-cbd9-4693-a35b-ba415cd837e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:53.378248 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f10fbf-cbd9-4693-a35b-ba415cd837e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:53.378402 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f10fbf-cbd9-4693-a35b-ba415cd837e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:53.378667 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f10fbf-cbd9-4693-a35b-ba415cd837e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:53.379335 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f10fbf-cbd9-4693-a35b-ba415cd837e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:53.380919 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f10fbf-cbd9-4693-a35b-ba415cd837e2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:53.392160 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1442/2851] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:26:53.396255 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371eeb1-fdf4-47e6-b943-774c9b0295ce admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:53.407127 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371eeb1-fdf4-47e6-b943-774c9b0295ce admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:53.411169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37652a77-10aa-4c04-8d9f-42ee7dcc6aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:53.411810 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37652a77-10aa-4c04-8d9f-42ee7dcc6aed admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:53.412088 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37652a77-10aa-4c04-8d9f-42ee7dcc6aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:53.412383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37652a77-10aa-4c04-8d9f-42ee7dcc6aed admin admin] PATH_INFO: `/v3/projects/b8ece0ed717d4eb0a7df42c2a5b76c34/users/34038f18afae4558bb56f4ed60ec80f7/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:53.427614 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37652a77-10aa-4c04-8d9f-42ee7dcc6aed admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8ece0ed717d4eb0a7df42c2a5b76c34, user_id=34038f18afae4558bb56f4ed60ec80f7, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:53.430277 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37652a77-10aa-4c04-8d9f-42ee7dcc6aed admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:53.546999 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1410/2852] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:26:53 2022] PUT /identity/v3/projects/b8ece0ed717d4eb0a7df42c2a5b76c34/users/34038f18afae4558bb56f4ed60ec80f7/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:53.551087 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f10fbf-cbd9-4693-a35b-ba415cd837e2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:53.561965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f10fbf-cbd9-4693-a35b-ba415cd837e2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:53.618120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002258fe-21b3-431c-9495-12cf62b30116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:53.618717 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002258fe-21b3-431c-9495-12cf62b30116 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:53.619023 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002258fe-21b3-431c-9495-12cf62b30116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:53.619289 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002258fe-21b3-431c-9495-12cf62b30116 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:53.619924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002258fe-21b3-431c-9495-12cf62b30116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:53.621509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002258fe-21b3-431c-9495-12cf62b30116 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:53.635541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1443/2853] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:26:53.636671 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37652a77-10aa-4c04-8d9f-42ee7dcc6aed admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:53.646929 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37652a77-10aa-4c04-8d9f-42ee7dcc6aed admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:53.651548 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2ff3f-a639-4baa-a8e7-3535f633e9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:53.652188 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2ff3f-a639-4baa-a8e7-3535f633e9ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:53.652457 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2ff3f-a639-4baa-a8e7-3535f633e9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:53.652725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2ff3f-a639-4baa-a8e7-3535f633e9ad admin admin] PATH_INFO: `/v3/projects/b8ece0ed717d4eb0a7df42c2a5b76c34/users/34038f18afae4558bb56f4ed60ec80f7/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:53.654794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b2ff3f-a639-4baa-a8e7-3535f633e9ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8ece0ed717d4eb0a7df42c2a5b76c34, user_id=34038f18afae4558bb56f4ed60ec80f7, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:53.657572 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b2ff3f-a639-4baa-a8e7-3535f633e9ad admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:53.667704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1411/2854] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:26:53 2022] PUT /identity/v3/projects/b8ece0ed717d4eb0a7df42c2a5b76c34/users/34038f18afae4558bb56f4ed60ec80f7/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:53.678004 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf0b23-39f7-436c-bc10-1d33484842c8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:53.678281 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf0b23-39f7-436c-bc10-1d33484842c8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:53.678553 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf0b23-39f7-436c-bc10-1d33484842c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:53.706912 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0ebf0b23-39f7-436c-bc10-1d33484842c8 None None] MFA Rules not processed for user `34038f18afae4558bb56f4ed60ec80f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:53.708138 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0ebf0b23-39f7-436c-bc10-1d33484842c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:53.716088 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:53.775272 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1444/2855] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:53.786834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2ff3f-a639-4baa-a8e7-3535f633e9ad admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:53.788566 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-56b2ff3f-a639-4baa-a8e7-3535f633e9ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:53.846017 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2ff3f-a639-4baa-a8e7-3535f633e9ad admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:53.849600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fad045-1876-4691-a2e9-af5171558b84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:53.850210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fad045-1876-4691-a2e9-af5171558b84 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:53.850486 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fad045-1876-4691-a2e9-af5171558b84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:53.850752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fad045-1876-4691-a2e9-af5171558b84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:53.878888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:53.905722 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fad045-1876-4691-a2e9-af5171558b84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:53.907221 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fad045-1876-4691-a2e9-af5171558b84 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:53.916224 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1412/2856] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:26:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:26:57.150998 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf0b23-39f7-436c-bc10-1d33484842c8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:57.215555 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf0b23-39f7-436c-bc10-1d33484842c8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:57.220406 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786ea62-101e-404b-8e3b-8cc8a567e783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:57.221060 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786ea62-101e-404b-8e3b-8cc8a567e783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:57.221334 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786ea62-101e-404b-8e3b-8cc8a567e783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:57.221603 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786ea62-101e-404b-8e3b-8cc8a567e783 admin admin] PATH_INFO: `/v3/users/0b5e03c3c60649778a8f68de20675015` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:57.222765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786ea62-101e-404b-8e3b-8cc8a567e783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b5e03c3c60649778a8f68de20675015)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:57.224759 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786ea62-101e-404b-8e3b-8cc8a567e783 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:57.429876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6786ea62-101e-404b-8e3b-8cc8a567e783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b5e03c3c60649778a8f68de20675015'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:57.442792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6786ea62-101e-404b-8e3b-8cc8a567e783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b5e03c3c60649778a8f68de20675015'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:57.443655 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6786ea62-101e-404b-8e3b-8cc8a567e783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b5e03c3c60649778a8f68de20675015'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:57.563645 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6786ea62-101e-404b-8e3b-8cc8a567e783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b5e03c3c60649778a8f68de20675015'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:57.564265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6786ea62-101e-404b-8e3b-8cc8a567e783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b5e03c3c60649778a8f68de20675015'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:57.577324 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1445/2857] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:26:57 2022] DELETE /identity/v3/users/0b5e03c3c60649778a8f68de20675015 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:57.581982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d685c-e116-49ed-9a3d-4177025e6a61 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:57.582436 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d685c-e116-49ed-9a3d-4177025e6a61 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:57.582651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d685c-e116-49ed-9a3d-4177025e6a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:57.602047 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-7d1d685c-e116-49ed-9a3d-4177025e6a61 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:57.603301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7d1d685c-e116-49ed-9a3d-4177025e6a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:57.661235 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1413/2858] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:57.671881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786ea62-101e-404b-8e3b-8cc8a567e783 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:57.673291 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6786ea62-101e-404b-8e3b-8cc8a567e783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:57.702159 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:57.731128 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786ea62-101e-404b-8e3b-8cc8a567e783 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:57.735786 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82141b5c-aa0c-4011-a36e-f8b3b4f681a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:57.736387 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82141b5c-aa0c-4011-a36e-f8b3b4f681a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:57.736663 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82141b5c-aa0c-4011-a36e-f8b3b4f681a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:57.736930 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82141b5c-aa0c-4011-a36e-f8b3b4f681a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:57.746325 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82141b5c-aa0c-4011-a36e-f8b3b4f681a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:57.748356 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82141b5c-aa0c-4011-a36e-f8b3b4f681a9 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:57.757838 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1446/2859] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:26:58.168708 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d685c-e116-49ed-9a3d-4177025e6a61 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:58.170217 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7d1d685c-e116-49ed-9a3d-4177025e6a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:58.196836 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d685c-e116-49ed-9a3d-4177025e6a61 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:58.203183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2146300-43cc-42ed-b7b1-1a930451e75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:58.204275 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2146300-43cc-42ed-b7b1-1a930451e75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:58.204395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2146300-43cc-42ed-b7b1-1a930451e75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:58.204731 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2146300-43cc-42ed-b7b1-1a930451e75f admin admin] PATH_INFO: `/v3/projects/98d174ba6a534f29b933594a420781dd` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:58.206124 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2146300-43cc-42ed-b7b1-1a930451e75f admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d174ba6a534f29b933594a420781dd)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:58.208522 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2146300-43cc-42ed-b7b1-1a930451e75f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:58.349470 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c2146300-43cc-42ed-b7b1-1a930451e75f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d174ba6a534f29b933594a420781dd', 'request_id': 'req-c2146300-43cc-42ed-b7b1-1a930451e75f'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:58.355907 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c2146300-43cc-42ed-b7b1-1a930451e75f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d174ba6a534f29b933594a420781dd', 'request_id': 'req-c2146300-43cc-42ed-b7b1-1a930451e75f'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:58.438110 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c2146300-43cc-42ed-b7b1-1a930451e75f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d174ba6a534f29b933594a420781dd', 'request_id': 'req-c2146300-43cc-42ed-b7b1-1a930451e75f'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:58.449139 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c2146300-43cc-42ed-b7b1-1a930451e75f admin admin] The token cache is being invalidate because project 98d174ba6a534f29b933594a420781dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:26:58.449431 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c2146300-43cc-42ed-b7b1-1a930451e75f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:58.450075 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c2146300-43cc-42ed-b7b1-1a930451e75f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:58.451816 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1414/2860] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:26:58 2022] DELETE /identity/v3/projects/98d174ba6a534f29b933594a420781dd => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:58.455934 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82141b5c-aa0c-4011-a36e-f8b3b4f681a9 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:58.465686 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82141b5c-aa0c-4011-a36e-f8b3b4f681a9 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:58.524760 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d050d-95b4-4138-8cc1-67c0df1b9e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:58.525389 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d050d-95b4-4138-8cc1-67c0df1b9e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:58.525666 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d050d-95b4-4138-8cc1-67c0df1b9e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:58.525938 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d050d-95b4-4138-8cc1-67c0df1b9e9e admin admin] PATH_INFO: `/v3/users/34038f18afae4558bb56f4ed60ec80f7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:58.527179 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d050d-95b4-4138-8cc1-67c0df1b9e9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=34038f18afae4558bb56f4ed60ec80f7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:58.535111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d050d-95b4-4138-8cc1-67c0df1b9e9e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:58.711259 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff6d050d-95b4-4138-8cc1-67c0df1b9e9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34038f18afae4558bb56f4ed60ec80f7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:58.724700 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff6d050d-95b4-4138-8cc1-67c0df1b9e9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34038f18afae4558bb56f4ed60ec80f7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:58.725635 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff6d050d-95b4-4138-8cc1-67c0df1b9e9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34038f18afae4558bb56f4ed60ec80f7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:58.807047 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff6d050d-95b4-4138-8cc1-67c0df1b9e9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34038f18afae4558bb56f4ed60ec80f7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:58.807792 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ff6d050d-95b4-4138-8cc1-67c0df1b9e9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34038f18afae4558bb56f4ed60ec80f7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:58.824773 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1447/2861] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:26:58 2022] DELETE /identity/v3/users/34038f18afae4558bb56f4ed60ec80f7 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:59.324923 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2146300-43cc-42ed-b7b1-1a930451e75f admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:59.327388 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-c2146300-43cc-42ed-b7b1-1a930451e75f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:59.339407 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2146300-43cc-42ed-b7b1-1a930451e75f admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:59.391595 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:59.400555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2ef9c-c70b-41cd-8c21-01a667a6b98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:59.401264 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2ef9c-c70b-41cd-8c21-01a667a6b98c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:59.401418 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2ef9c-c70b-41cd-8c21-01a667a6b98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:59.401827 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2ef9c-c70b-41cd-8c21-01a667a6b98c admin admin] PATH_INFO: `/v3/projects/b8ece0ed717d4eb0a7df42c2a5b76c34` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:59.402971 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2ef9c-c70b-41cd-8c21-01a667a6b98c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8ece0ed717d4eb0a7df42c2a5b76c34)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:59.407331 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2ef9c-c70b-41cd-8c21-01a667a6b98c admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:59.530095 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bfb2ef9c-c70b-41cd-8c21-01a667a6b98c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8ece0ed717d4eb0a7df42c2a5b76c34', 'request_id': 'req-bfb2ef9c-c70b-41cd-8c21-01a667a6b98c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:59.536866 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bfb2ef9c-c70b-41cd-8c21-01a667a6b98c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8ece0ed717d4eb0a7df42c2a5b76c34', 'request_id': 'req-bfb2ef9c-c70b-41cd-8c21-01a667a6b98c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:59.606883 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bfb2ef9c-c70b-41cd-8c21-01a667a6b98c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8ece0ed717d4eb0a7df42c2a5b76c34', 'request_id': 'req-bfb2ef9c-c70b-41cd-8c21-01a667a6b98c'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:59.617878 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bfb2ef9c-c70b-41cd-8c21-01a667a6b98c admin admin] The token cache is being invalidate because project b8ece0ed717d4eb0a7df42c2a5b76c34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:26:59.618167 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bfb2ef9c-c70b-41cd-8c21-01a667a6b98c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:59.618818 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-bfb2ef9c-c70b-41cd-8c21-01a667a6b98c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:26:59.620335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1415/2862] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:26:59 2022] DELETE /identity/v3/projects/b8ece0ed717d4eb0a7df42c2a5b76c34 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:26:59.625508 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7671012-514e-4c9d-b10e-2afdc51c72d1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:59.630843 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7671012-514e-4c9d-b10e-2afdc51c72d1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:59.630843 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7671012-514e-4c9d-b10e-2afdc51c72d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:59.647812 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f7671012-514e-4c9d-b10e-2afdc51c72d1 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:59.649027 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f7671012-514e-4c9d-b10e-2afdc51c72d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:59.691239 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:26:59.708664 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1448/2863] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:26:59.718116 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df7ecc-6b02-40cf-92b3-a7ce3cb01fc3 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:59.718396 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df7ecc-6b02-40cf-92b3-a7ce3cb01fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:59.718669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df7ecc-6b02-40cf-92b3-a7ce3cb01fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:59.738833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-47df7ecc-6b02-40cf-92b3-a7ce3cb01fc3 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:26:59.740052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-47df7ecc-6b02-40cf-92b3-a7ce3cb01fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:26:59.748582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1416/2864] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 06 08:26:59.752859 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7671012-514e-4c9d-b10e-2afdc51c72d1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:59.762573 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7671012-514e-4c9d-b10e-2afdc51c72d1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:59.767681 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e6a60-d044-478c-872a-5c240ee5d6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:59.768296 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e6a60-d044-478c-872a-5c240ee5d6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:59.771409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e6a60-d044-478c-872a-5c240ee5d6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:59.771409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e6a60-d044-478c-872a-5c240ee5d6c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:59.771409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e6a60-d044-478c-872a-5c240ee5d6c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:26:59.771409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e6a60-d044-478c-872a-5c240ee5d6c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:59.771832 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e6a60-d044-478c-872a-5c240ee5d6c0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:59.787033 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1449/2865] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 08:26:59.788186 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df7ecc-6b02-40cf-92b3-a7ce3cb01fc3 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:59.797901 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df7ecc-6b02-40cf-92b3-a7ce3cb01fc3 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:59.802522 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e3336-065d-4d53-9875-0861ec453f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:59.803136 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e3336-065d-4d53-9875-0861ec453f7d admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:59.803408 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e3336-065d-4d53-9875-0861ec453f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:59.803672 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e3336-065d-4d53-9875-0861ec453f7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:59.804292 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e3336-065d-4d53-9875-0861ec453f7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:59.806235 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e3336-065d-4d53-9875-0861ec453f7d admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:26:59.882813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1417/2866] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:26:59 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:26:59.886899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e6a60-d044-478c-872a-5c240ee5d6c0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:26:59.897156 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e6a60-d044-478c-872a-5c240ee5d6c0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:26:59.953038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6d33a-8b65-41f5-b3dc-3f4a0be6da99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:26:59.953635 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6d33a-8b65-41f5-b3dc-3f4a0be6da99 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:26:59.953906 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6d33a-8b65-41f5-b3dc-3f4a0be6da99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:26:59.954171 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6d33a-8b65-41f5-b3dc-3f4a0be6da99 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:26:59.954788 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6d33a-8b65-41f5-b3dc-3f4a0be6da99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:26:59.956729 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6d33a-8b65-41f5-b3dc-3f4a0be6da99 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:00.033336 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1450/2867] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:26:59 2022] POST /identity/v3/users => generated 291 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:27:00.037523 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e3336-065d-4d53-9875-0861ec453f7d admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:00.048944 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e3336-065d-4d53-9875-0861ec453f7d admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:00.053861 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee8cf3-8d07-4f68-b86a-e713dd1d523e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:00.054448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee8cf3-8d07-4f68-b86a-e713dd1d523e admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:00.054721 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee8cf3-8d07-4f68-b86a-e713dd1d523e admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:00.055023 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee8cf3-8d07-4f68-b86a-e713dd1d523e admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:00.055663 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee8cf3-8d07-4f68-b86a-e713dd1d523e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:00.057253 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee8cf3-8d07-4f68-b86a-e713dd1d523e admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:00.067447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1418/2868] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:27:00.071480 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6d33a-8b65-41f5-b3dc-3f4a0be6da99 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:00.081490 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6d33a-8b65-41f5-b3dc-3f4a0be6da99 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:00.086046 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c2683-30e9-4ce3-b185-6fdd6d56496d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:00.086624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c2683-30e9-4ce3-b185-6fdd6d56496d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:00.086897 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c2683-30e9-4ce3-b185-6fdd6d56496d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:00.087190 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c2683-30e9-4ce3-b185-6fdd6d56496d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:00.087819 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c2683-30e9-4ce3-b185-6fdd6d56496d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:00.089390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c2683-30e9-4ce3-b185-6fdd6d56496d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:00.100918 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1451/2869] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:27:00.104928 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee8cf3-8d07-4f68-b86a-e713dd1d523e admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:00.115483 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee8cf3-8d07-4f68-b86a-e713dd1d523e admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:00.120370 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924f35d-ff7e-4c80-9180-4db805c1d416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:00.120998 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f35d-ff7e-4c80-9180-4db805c1d416 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:00.121268 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f35d-ff7e-4c80-9180-4db805c1d416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:00.121532 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f35d-ff7e-4c80-9180-4db805c1d416 admin admin] PATH_INFO: `/v3/projects/ccd518dad30a4ba0a0d752ebbca9633b/users/7d2d5435d79049b1b089f611309d65fc/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:00.136169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924f35d-ff7e-4c80-9180-4db805c1d416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccd518dad30a4ba0a0d752ebbca9633b, user_id=7d2d5435d79049b1b089f611309d65fc, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:00.138704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924f35d-ff7e-4c80-9180-4db805c1d416 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:00.219719 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1419/2870] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:27:00 2022] PUT /identity/v3/projects/ccd518dad30a4ba0a0d752ebbca9633b/users/7d2d5435d79049b1b089f611309d65fc/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:27:00.230092 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632f400-d1ca-4bb7-82ba-11b7b7daae3d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:00.230407 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632f400-d1ca-4bb7-82ba-11b7b7daae3d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:00.230679 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632f400-d1ca-4bb7-82ba-11b7b7daae3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:00.260080 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3632f400-d1ca-4bb7-82ba-11b7b7daae3d None None] MFA Rules not processed for user `7d2d5435d79049b1b089f611309d65fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:27:00.261321 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3632f400-d1ca-4bb7-82ba-11b7b7daae3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:00.325769 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1452/2871] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:27:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:27:00.330410 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924f35d-ff7e-4c80-9180-4db805c1d416 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:00.393378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924f35d-ff7e-4c80-9180-4db805c1d416 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:00.398063 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c7414-7da0-48e3-a5e5-7a345b0c48a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:00.398669 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c7414-7da0-48e3-a5e5-7a345b0c48a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:00.398959 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c7414-7da0-48e3-a5e5-7a345b0c48a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:00.399246 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c7414-7da0-48e3-a5e5-7a345b0c48a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:00.399882 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c7414-7da0-48e3-a5e5-7a345b0c48a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:00.401789 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c7414-7da0-48e3-a5e5-7a345b0c48a0 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:00.495582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1420/2872] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:27:00 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:27:00.500721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632f400-d1ca-4bb7-82ba-11b7b7daae3d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:00.511133 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632f400-d1ca-4bb7-82ba-11b7b7daae3d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:00.551198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:00.570211 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0318f-4882-4f5b-8f80-702329d00139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:00.570211 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0318f-4882-4f5b-8f80-702329d00139 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:00.570211 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0318f-4882-4f5b-8f80-702329d00139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:00.570211 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0318f-4882-4f5b-8f80-702329d00139 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:00.570577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0318f-4882-4f5b-8f80-702329d00139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:00.570577 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0318f-4882-4f5b-8f80-702329d00139 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:00.647948 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1453/2873] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:27:00 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:27:00.651844 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c7414-7da0-48e3-a5e5-7a345b0c48a0 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:00.661331 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:00.671792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c7414-7da0-48e3-a5e5-7a345b0c48a0 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:00.676630 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29766045-d33a-4893-9096-66d28239df6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:00.684888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29766045-d33a-4893-9096-66d28239df6e admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:00.684888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29766045-d33a-4893-9096-66d28239df6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:00.684888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29766045-d33a-4893-9096-66d28239df6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:00.684888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29766045-d33a-4893-9096-66d28239df6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:00.684888 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29766045-d33a-4893-9096-66d28239df6e admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:00.691695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1421/2874] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:27:00.696030 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0318f-4882-4f5b-8f80-702329d00139 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:00.708102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0318f-4882-4f5b-8f80-702329d00139 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:00.717218 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629de92e-4724-4f51-9e20-9040c8fa7370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:00.717944 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629de92e-4724-4f51-9e20-9040c8fa7370 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:00.718198 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629de92e-4724-4f51-9e20-9040c8fa7370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:00.718471 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629de92e-4724-4f51-9e20-9040c8fa7370 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:00.719153 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629de92e-4724-4f51-9e20-9040c8fa7370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:00.725203 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629de92e-4724-4f51-9e20-9040c8fa7370 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:00.733975 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1454/2875] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:27:00.737992 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29766045-d33a-4893-9096-66d28239df6e admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:00.748390 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29766045-d33a-4893-9096-66d28239df6e admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:00.753037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3258ad-c8b6-426a-98da-7de242a86534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:00.753676 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3258ad-c8b6-426a-98da-7de242a86534 admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:00.753949 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3258ad-c8b6-426a-98da-7de242a86534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:00.754218 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3258ad-c8b6-426a-98da-7de242a86534 admin admin] PATH_INFO: `/v3/projects/32ebf2f83e8e43df918de5e90c5f362a/users/c84b2ef96c604ea6a298cde3f7320036/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:00.769356 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3258ad-c8b6-426a-98da-7de242a86534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32ebf2f83e8e43df918de5e90c5f362a, user_id=c84b2ef96c604ea6a298cde3f7320036, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:00.772041 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3258ad-c8b6-426a-98da-7de242a86534 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:00.861052 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1422/2876] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:27:00 2022] PUT /identity/v3/projects/32ebf2f83e8e43df918de5e90c5f362a/users/c84b2ef96c604ea6a298cde3f7320036/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:27:00.873152 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe22a5f-3578-4437-816f-dc6aaf2e1fb4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:00.873337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe22a5f-3578-4437-816f-dc6aaf2e1fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:00.873592 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe22a5f-3578-4437-816f-dc6aaf2e1fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:00.902401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1fe22a5f-3578-4437-816f-dc6aaf2e1fb4 None None] MFA Rules not processed for user `c84b2ef96c604ea6a298cde3f7320036`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:27:00.903731 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1fe22a5f-3578-4437-816f-dc6aaf2e1fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:00.967877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1455/2877] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:27:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:27:00.972180 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3258ad-c8b6-426a-98da-7de242a86534 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:01.035131 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3258ad-c8b6-426a-98da-7de242a86534 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:01.039853 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29eb8f3-6711-45a0-81c8-6fb1bee6d40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:01.040450 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29eb8f3-6711-45a0-81c8-6fb1bee6d40c admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:01.040726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29eb8f3-6711-45a0-81c8-6fb1bee6d40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:01.040995 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29eb8f3-6711-45a0-81c8-6fb1bee6d40c admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:01.041637 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29eb8f3-6711-45a0-81c8-6fb1bee6d40c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:01.043662 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29eb8f3-6711-45a0-81c8-6fb1bee6d40c admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:01.118854 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1423/2878] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:27:00 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:27:01.120091 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe22a5f-3578-4437-816f-dc6aaf2e1fb4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:01.132038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe22a5f-3578-4437-816f-dc6aaf2e1fb4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:01.188876 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca78b9e-1215-4548-8dc2-9cfa54efa1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:01.189473 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca78b9e-1215-4548-8dc2-9cfa54efa1ef admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:01.189746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca78b9e-1215-4548-8dc2-9cfa54efa1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:01.190013 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca78b9e-1215-4548-8dc2-9cfa54efa1ef admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:01.190636 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca78b9e-1215-4548-8dc2-9cfa54efa1ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:01.192609 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca78b9e-1215-4548-8dc2-9cfa54efa1ef admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:01.301268 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1456/2879] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:27:01 2022] POST /identity/v3/users => generated 299 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:27:01.305413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29eb8f3-6711-45a0-81c8-6fb1bee6d40c admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:01.315817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29eb8f3-6711-45a0-81c8-6fb1bee6d40c admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:01.320914 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb2472-ef8c-45e2-86fd-4374309b8fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:01.322198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb2472-ef8c-45e2-86fd-4374309b8fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:01.322198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb2472-ef8c-45e2-86fd-4374309b8fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:01.322198 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb2472-ef8c-45e2-86fd-4374309b8fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:01.322748 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb2472-ef8c-45e2-86fd-4374309b8fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:01.324432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb2472-ef8c-45e2-86fd-4374309b8fb7 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:01.335092 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1424/2880] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:27:01.339184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca78b9e-1215-4548-8dc2-9cfa54efa1ef admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:01.350298 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca78b9e-1215-4548-8dc2-9cfa54efa1ef admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:01.354853 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625da70-8b0a-477d-8017-f5b71161898d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:01.355510 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625da70-8b0a-477d-8017-f5b71161898d admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:01.355785 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625da70-8b0a-477d-8017-f5b71161898d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:01.356057 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625da70-8b0a-477d-8017-f5b71161898d admin admin] PATH_INFO: `/v3/projects/8ea3c1ea759846bea30df1b9a5f49214/users/caa1ecca8cfb45ea80d6bbf6db8332a7/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:01.371957 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625da70-8b0a-477d-8017-f5b71161898d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ea3c1ea759846bea30df1b9a5f49214, user_id=caa1ecca8cfb45ea80d6bbf6db8332a7, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:01.374861 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625da70-8b0a-477d-8017-f5b71161898d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:01.466380 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1457/2881] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:27:01 2022] PUT /identity/v3/projects/8ea3c1ea759846bea30df1b9a5f49214/users/caa1ecca8cfb45ea80d6bbf6db8332a7/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:27:01.470442 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb2472-ef8c-45e2-86fd-4374309b8fb7 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:01.480704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb2472-ef8c-45e2-86fd-4374309b8fb7 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:01.537152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3497f-4c6b-4266-871a-5a3d12e4c21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:01.537744 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3497f-4c6b-4266-871a-5a3d12e4c21b admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:01.538022 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3497f-4c6b-4266-871a-5a3d12e4c21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:01.538293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3497f-4c6b-4266-871a-5a3d12e4c21b admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:01.538922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3497f-4c6b-4266-871a-5a3d12e4c21b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:01.540541 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3497f-4c6b-4266-871a-5a3d12e4c21b admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:01.550647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1425/2882] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:27:01.554771 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625da70-8b0a-477d-8017-f5b71161898d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:01.565627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625da70-8b0a-477d-8017-f5b71161898d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:01.570183 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17029b-c2c5-4644-a7c3-850e39b60c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:01.570823 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17029b-c2c5-4644-a7c3-850e39b60c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:01.571130 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17029b-c2c5-4644-a7c3-850e39b60c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:01.571403 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17029b-c2c5-4644-a7c3-850e39b60c66 admin admin] PATH_INFO: `/v3/projects/8ea3c1ea759846bea30df1b9a5f49214/users/caa1ecca8cfb45ea80d6bbf6db8332a7/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:01.573311 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17029b-c2c5-4644-a7c3-850e39b60c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ea3c1ea759846bea30df1b9a5f49214, user_id=caa1ecca8cfb45ea80d6bbf6db8332a7, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:01.576411 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17029b-c2c5-4644-a7c3-850e39b60c66 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:01.586458 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1458/2883] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:27:01 2022] PUT /identity/v3/projects/8ea3c1ea759846bea30df1b9a5f49214/users/caa1ecca8cfb45ea80d6bbf6db8332a7/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:27:01.597560 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4157558-02db-4339-b478-bb32e5c73329 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:01.597839 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4157558-02db-4339-b478-bb32e5c73329 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:01.598111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4157558-02db-4339-b478-bb32e5c73329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:01.627383 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-a4157558-02db-4339-b478-bb32e5c73329 None None] MFA Rules not processed for user `caa1ecca8cfb45ea80d6bbf6db8332a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:27:01.647506 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a4157558-02db-4339-b478-bb32e5c73329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:01.657285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:01.696258 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1426/2884] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:27:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:27:01.708693 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17029b-c2c5-4644-a7c3-850e39b60c66 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:01.709742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-be17029b-c2c5-4644-a7c3-850e39b60c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:01.715033 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:01.776136 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17029b-c2c5-4644-a7c3-850e39b60c66 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:01.783168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf4c2e-febc-4be5-84c6-aecfd540ed53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:01.783168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf4c2e-febc-4be5-84c6-aecfd540ed53 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:01.783168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf4c2e-febc-4be5-84c6-aecfd540ed53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:01.783168 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf4c2e-febc-4be5-84c6-aecfd540ed53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:01.840419 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf4c2e-febc-4be5-84c6-aecfd540ed53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:01.841922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf4c2e-febc-4be5-84c6-aecfd540ed53 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:01.851021 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1459/2885] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:27:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:27:07.961159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4157558-02db-4339-b478-bb32e5c73329 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:07.971386 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4157558-02db-4339-b478-bb32e5c73329 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:07.974903 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f7256-357c-4881-a16c-0f34b475ad94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:07.975519 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f7256-357c-4881-a16c-0f34b475ad94 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:07.975790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f7256-357c-4881-a16c-0f34b475ad94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:07.976054 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f7256-357c-4881-a16c-0f34b475ad94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:08.030996 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f7256-357c-4881-a16c-0f34b475ad94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:08.032417 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f7256-357c-4881-a16c-0f34b475ad94 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:08.041192 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1427/2886] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:27:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:27:09.494852 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf4c2e-febc-4be5-84c6-aecfd540ed53 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:09.496338 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4eaf4c2e-febc-4be5-84c6-aecfd540ed53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:09.507818 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf4c2e-febc-4be5-84c6-aecfd540ed53 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:09.511553 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97234361-4833-414e-883c-32fca918d03e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:09.512155 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97234361-4833-414e-883c-32fca918d03e service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:09.512435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97234361-4833-414e-883c-32fca918d03e service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:09.513359 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97234361-4833-414e-883c-32fca918d03e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:09.523053 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97234361-4833-414e-883c-32fca918d03e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:09.524453 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97234361-4833-414e-883c-32fca918d03e service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:09.535093 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1460/2887] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:27:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:27:10.482649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f7256-357c-4881-a16c-0f34b475ad94 service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:10.484194 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-aa2f7256-357c-4881-a16c-0f34b475ad94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:10.557873 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f7256-357c-4881-a16c-0f34b475ad94 service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:10.564413 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa435e-8350-4121-a512-85cbc2df80c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:10.565028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa435e-8350-4121-a512-85cbc2df80c9 service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:10.565307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa435e-8350-4121-a512-85cbc2df80c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:10.565580 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa435e-8350-4121-a512-85cbc2df80c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:10.567416 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-edaa435e-8350-4121-a512-85cbc2df80c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:10.581943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa435e-8350-4121-a512-85cbc2df80c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:10.583665 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa435e-8350-4121-a512-85cbc2df80c9 service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:10.594834 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1428/2888] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:27:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:27:12.085723 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97234361-4833-414e-883c-32fca918d03e service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:12.087210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-97234361-4833-414e-883c-32fca918d03e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:12.151822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97234361-4833-414e-883c-32fca918d03e service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:12.156625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2879822-320a-4d67-8a1c-a830afbd0a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:12.157322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2879822-320a-4d67-8a1c-a830afbd0a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:12.157656 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2879822-320a-4d67-8a1c-a830afbd0a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:12.157952 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2879822-320a-4d67-8a1c-a830afbd0a40 admin admin] PATH_INFO: `/v3/users/a1823fec359f4b1c8b2ad6360b1bc6e3` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:12.159102 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2879822-320a-4d67-8a1c-a830afbd0a40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1823fec359f4b1c8b2ad6360b1bc6e3)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:12.161063 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2879822-320a-4d67-8a1c-a830afbd0a40 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:12.326318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:12.328184 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c2879822-320a-4d67-8a1c-a830afbd0a40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1823fec359f4b1c8b2ad6360b1bc6e3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:12.341051 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c2879822-320a-4d67-8a1c-a830afbd0a40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1823fec359f4b1c8b2ad6360b1bc6e3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:12.341883 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c2879822-320a-4d67-8a1c-a830afbd0a40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1823fec359f4b1c8b2ad6360b1bc6e3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:12.452541 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c2879822-320a-4d67-8a1c-a830afbd0a40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1823fec359f4b1c8b2ad6360b1bc6e3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:12.453853 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c2879822-320a-4d67-8a1c-a830afbd0a40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1823fec359f4b1c8b2ad6360b1bc6e3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:12.469326 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1461/2889] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:27:12 2022] DELETE /identity/v3/users/a1823fec359f4b1c8b2ad6360b1bc6e3 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:27:12.473328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e8754-d795-44ea-ae9b-13c141c91f0a None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:12.473813 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e8754-d795-44ea-ae9b-13c141c91f0a None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:12.473943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e8754-d795-44ea-ae9b-13c141c91f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:12.497033 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-6f3e8754-d795-44ea-ae9b-13c141c91f0a None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:27:12.497033 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6f3e8754-d795-44ea-ae9b-13c141c91f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:12.549446 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=100 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:12.570076 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1429/2890] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:27:12.580270 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2879822-320a-4d67-8a1c-a830afbd0a40 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:12.581648 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c2879822-320a-4d67-8a1c-a830afbd0a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:12.641010 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2879822-320a-4d67-8a1c-a830afbd0a40 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:12.644736 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a515049-bd92-45b6-8259-d4f3410eb079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:12.645426 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a515049-bd92-45b6-8259-d4f3410eb079 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:12.645637 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a515049-bd92-45b6-8259-d4f3410eb079 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:12.645900 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a515049-bd92-45b6-8259-d4f3410eb079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:12.656808 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a515049-bd92-45b6-8259-d4f3410eb079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:12.658270 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a515049-bd92-45b6-8259-d4f3410eb079 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:12.668234 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1462/2891] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:27:13.165726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e8754-d795-44ea-ae9b-13c141c91f0a None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:13.165726 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-6f3e8754-d795-44ea-ae9b-13c141c91f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:13.171343 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e8754-d795-44ea-ae9b-13c141c91f0a None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:13.176212 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4d232-c6ff-4413-8012-5945e1819233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:13.176848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4d232-c6ff-4413-8012-5945e1819233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:13.177328 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4d232-c6ff-4413-8012-5945e1819233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:13.177625 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4d232-c6ff-4413-8012-5945e1819233 admin admin] PATH_INFO: `/v3/projects/3e20d4ae8a634548bfd9540de8ab5653` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:13.178751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4d232-c6ff-4413-8012-5945e1819233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e20d4ae8a634548bfd9540de8ab5653)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:13.180723 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4d232-c6ff-4413-8012-5945e1819233 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:13.333629 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c8e4d232-c6ff-4413-8012-5945e1819233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e20d4ae8a634548bfd9540de8ab5653', 'request_id': 'req-c8e4d232-c6ff-4413-8012-5945e1819233'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:13.340090 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c8e4d232-c6ff-4413-8012-5945e1819233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e20d4ae8a634548bfd9540de8ab5653', 'request_id': 'req-c8e4d232-c6ff-4413-8012-5945e1819233'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:13.420727 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c8e4d232-c6ff-4413-8012-5945e1819233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e20d4ae8a634548bfd9540de8ab5653', 'request_id': 'req-c8e4d232-c6ff-4413-8012-5945e1819233'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:13.431101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c8e4d232-c6ff-4413-8012-5945e1819233 admin admin] The token cache is being invalidate because project 3e20d4ae8a634548bfd9540de8ab5653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:27:13.431384 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c8e4d232-c6ff-4413-8012-5945e1819233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:13.431994 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-c8e4d232-c6ff-4413-8012-5945e1819233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:13.433454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1430/2892] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:27:13 2022] DELETE /identity/v3/projects/3e20d4ae8a634548bfd9540de8ab5653 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:27:13.887758 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a515049-bd92-45b6-8259-d4f3410eb079 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:13.897283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a515049-bd92-45b6-8259-d4f3410eb079 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:13.945092 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78741ff-189d-40e6-8dea-f42b51a1c72f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:13.945644 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78741ff-189d-40e6-8dea-f42b51a1c72f service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:13.945918 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78741ff-189d-40e6-8dea-f42b51a1c72f service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:13.946206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78741ff-189d-40e6-8dea-f42b51a1c72f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:13.947982 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f78741ff-189d-40e6-8dea-f42b51a1c72f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:14.012092 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78741ff-189d-40e6-8dea-f42b51a1c72f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:14.013532 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78741ff-189d-40e6-8dea-f42b51a1c72f service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:14.023924 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1463/2893] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:27:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:27:14.343968 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4d232-c6ff-4413-8012-5945e1819233 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:14.345464 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-c8e4d232-c6ff-4413-8012-5945e1819233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:14.413852 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4d232-c6ff-4413-8012-5945e1819233 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:14.418649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b386a4-a69c-49be-8fac-1cae13dbe1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:14.419308 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b386a4-a69c-49be-8fac-1cae13dbe1da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:14.419590 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b386a4-a69c-49be-8fac-1cae13dbe1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:14.419872 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b386a4-a69c-49be-8fac-1cae13dbe1da admin admin] PATH_INFO: `/v3/users/7d2d5435d79049b1b089f611309d65fc` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:14.421779 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b386a4-a69c-49be-8fac-1cae13dbe1da admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d2d5435d79049b1b089f611309d65fc)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:14.423799 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b386a4-a69c-49be-8fac-1cae13dbe1da admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:14.692431 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b5b386a4-a69c-49be-8fac-1cae13dbe1da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d2d5435d79049b1b089f611309d65fc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:14.693343 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b5b386a4-a69c-49be-8fac-1cae13dbe1da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d2d5435d79049b1b089f611309d65fc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:14.709102 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b5b386a4-a69c-49be-8fac-1cae13dbe1da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d2d5435d79049b1b089f611309d65fc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:14.781686 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b5b386a4-a69c-49be-8fac-1cae13dbe1da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d2d5435d79049b1b089f611309d65fc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:14.782311 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-b5b386a4-a69c-49be-8fac-1cae13dbe1da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d2d5435d79049b1b089f611309d65fc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:14.784725 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:14.797374 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1431/2894] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:27:14 2022] DELETE /identity/v3/users/7d2d5435d79049b1b089f611309d65fc => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:27:14.801765 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276849c-b773-4711-9ad0-3c0af15e300e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:14.802045 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276849c-b773-4711-9ad0-3c0af15e300e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:14.802342 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276849c-b773-4711-9ad0-3c0af15e300e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:14.829835 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9276849c-b773-4711-9ad0-3c0af15e300e None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:27:14.831073 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9276849c-b773-4711-9ad0-3c0af15e300e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:14.841097 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:14.890797 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1464/2895] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:27:14.902492 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b386a4-a69c-49be-8fac-1cae13dbe1da admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:14.904204 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b5b386a4-a69c-49be-8fac-1cae13dbe1da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:14.961879 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b386a4-a69c-49be-8fac-1cae13dbe1da admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:14.965448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986137a-b1a6-41ed-bf5d-6eaf4b65b58f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:14.966046 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986137a-b1a6-41ed-bf5d-6eaf4b65b58f service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:14.966323 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986137a-b1a6-41ed-bf5d-6eaf4b65b58f service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:14.967616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986137a-b1a6-41ed-bf5d-6eaf4b65b58f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:14.975745 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986137a-b1a6-41ed-bf5d-6eaf4b65b58f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:14.977190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986137a-b1a6-41ed-bf5d-6eaf4b65b58f service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:14.987121 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1432/2896] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:27:15.408460 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276849c-b773-4711-9ad0-3c0af15e300e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:15.410327 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9276849c-b773-4711-9ad0-3c0af15e300e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:15.425388 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276849c-b773-4711-9ad0-3c0af15e300e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:15.430066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0afa6c-1096-4317-baaa-125b3a287358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:15.431922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0afa6c-1096-4317-baaa-125b3a287358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:15.431922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0afa6c-1096-4317-baaa-125b3a287358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:15.431922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0afa6c-1096-4317-baaa-125b3a287358 admin admin] PATH_INFO: `/v3/projects/ccd518dad30a4ba0a0d752ebbca9633b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:15.440157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0afa6c-1096-4317-baaa-125b3a287358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccd518dad30a4ba0a0d752ebbca9633b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:15.442171 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0afa6c-1096-4317-baaa-125b3a287358 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:15.568747 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2d0afa6c-1096-4317-baaa-125b3a287358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccd518dad30a4ba0a0d752ebbca9633b', 'request_id': 'req-2d0afa6c-1096-4317-baaa-125b3a287358'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:15.581409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2d0afa6c-1096-4317-baaa-125b3a287358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccd518dad30a4ba0a0d752ebbca9633b', 'request_id': 'req-2d0afa6c-1096-4317-baaa-125b3a287358'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:15.643413 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2d0afa6c-1096-4317-baaa-125b3a287358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccd518dad30a4ba0a0d752ebbca9633b', 'request_id': 'req-2d0afa6c-1096-4317-baaa-125b3a287358'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:15.657111 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2d0afa6c-1096-4317-baaa-125b3a287358 admin admin] The token cache is being invalidate because project ccd518dad30a4ba0a0d752ebbca9633b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:27:15.657404 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2d0afa6c-1096-4317-baaa-125b3a287358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:15.658068 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2d0afa6c-1096-4317-baaa-125b3a287358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:15.659707 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1465/2897] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:27:15 2022] DELETE /identity/v3/projects/ccd518dad30a4ba0a0d752ebbca9633b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:27:15.663794 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986137a-b1a6-41ed-bf5d-6eaf4b65b58f service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:15.674288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986137a-b1a6-41ed-bf5d-6eaf4b65b58f service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:15.737021 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907955b0-8b33-4d95-880c-8ad8c0aabe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:15.737667 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907955b0-8b33-4d95-880c-8ad8c0aabe0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:15.737943 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907955b0-8b33-4d95-880c-8ad8c0aabe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:15.738210 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907955b0-8b33-4d95-880c-8ad8c0aabe0a admin admin] PATH_INFO: `/v3/users/c84b2ef96c604ea6a298cde3f7320036` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:15.739616 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907955b0-8b33-4d95-880c-8ad8c0aabe0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c84b2ef96c604ea6a298cde3f7320036)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:15.745910 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907955b0-8b33-4d95-880c-8ad8c0aabe0a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:15.943752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-907955b0-8b33-4d95-880c-8ad8c0aabe0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c84b2ef96c604ea6a298cde3f7320036'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:15.944406 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-907955b0-8b33-4d95-880c-8ad8c0aabe0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c84b2ef96c604ea6a298cde3f7320036'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:15.960399 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-907955b0-8b33-4d95-880c-8ad8c0aabe0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c84b2ef96c604ea6a298cde3f7320036'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:16.032226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-907955b0-8b33-4d95-880c-8ad8c0aabe0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c84b2ef96c604ea6a298cde3f7320036'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:16.032990 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-907955b0-8b33-4d95-880c-8ad8c0aabe0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c84b2ef96c604ea6a298cde3f7320036'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:16.048981 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1433/2898] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:27:15 2022] DELETE /identity/v3/users/c84b2ef96c604ea6a298cde3f7320036 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:27:16.384385 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0afa6c-1096-4317-baaa-125b3a287358 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:16.386207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2d0afa6c-1096-4317-baaa-125b3a287358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:16.475238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0afa6c-1096-4317-baaa-125b3a287358 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:16.509830 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:16.541211 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907955b0-8b33-4d95-880c-8ad8c0aabe0a admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:16.542865 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-907955b0-8b33-4d95-880c-8ad8c0aabe0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:16.549622 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2e262-f780-47b6-886e-a7e68726733e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:16.550280 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2e262-f780-47b6-886e-a7e68726733e service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:16.550579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2e262-f780-47b6-886e-a7e68726733e service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:16.550880 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2e262-f780-47b6-886e-a7e68726733e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:16.552809 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e3e2e262-f780-47b6-886e-a7e68726733e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:16.570240 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2e262-f780-47b6-886e-a7e68726733e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:16.571926 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2e262-f780-47b6-886e-a7e68726733e service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:16.580265 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:16.586210 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1466/2899] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:27:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:27:16.635151 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907955b0-8b33-4d95-880c-8ad8c0aabe0a admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:16.639999 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70c20d-555f-4c13-8306-c16f10a9bd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:16.641004 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70c20d-555f-4c13-8306-c16f10a9bd12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:16.641277 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70c20d-555f-4c13-8306-c16f10a9bd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:16.641546 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70c20d-555f-4c13-8306-c16f10a9bd12 admin admin] PATH_INFO: `/v3/projects/32ebf2f83e8e43df918de5e90c5f362a` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:16.642672 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b70c20d-555f-4c13-8306-c16f10a9bd12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32ebf2f83e8e43df918de5e90c5f362a)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:16.644615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b70c20d-555f-4c13-8306-c16f10a9bd12 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:16.759161 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-0b70c20d-555f-4c13-8306-c16f10a9bd12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32ebf2f83e8e43df918de5e90c5f362a', 'request_id': 'req-0b70c20d-555f-4c13-8306-c16f10a9bd12'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:16.766454 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-0b70c20d-555f-4c13-8306-c16f10a9bd12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32ebf2f83e8e43df918de5e90c5f362a', 'request_id': 'req-0b70c20d-555f-4c13-8306-c16f10a9bd12'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:16.825989 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-0b70c20d-555f-4c13-8306-c16f10a9bd12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32ebf2f83e8e43df918de5e90c5f362a', 'request_id': 'req-0b70c20d-555f-4c13-8306-c16f10a9bd12'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:16.837695 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-0b70c20d-555f-4c13-8306-c16f10a9bd12 admin admin] The token cache is being invalidate because project 32ebf2f83e8e43df918de5e90c5f362a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:27:16.837988 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-0b70c20d-555f-4c13-8306-c16f10a9bd12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:16.838647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-0b70c20d-555f-4c13-8306-c16f10a9bd12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:16.840394 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1434/2900] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:27:16 2022] DELETE /identity/v3/projects/32ebf2f83e8e43df918de5e90c5f362a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:27:16.844318 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2e262-f780-47b6-886e-a7e68726733e service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:16.854721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2e262-f780-47b6-886e-a7e68726733e service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:16.917381 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98de5f-b1cd-4ce8-8bc6-61b11f0bd22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:16.918206 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98de5f-b1cd-4ce8-8bc6-61b11f0bd22e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:16.918602 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98de5f-b1cd-4ce8-8bc6-61b11f0bd22e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:16.919008 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98de5f-b1cd-4ce8-8bc6-61b11f0bd22e admin admin] PATH_INFO: `/v3/users/caa1ecca8cfb45ea80d6bbf6db8332a7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:16.920847 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98de5f-b1cd-4ce8-8bc6-61b11f0bd22e admin admin] RBAC: Authorizing `identity:delete_user(user_id=caa1ecca8cfb45ea80d6bbf6db8332a7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:16.922899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98de5f-b1cd-4ce8-8bc6-61b11f0bd22e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:17.110417 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-aa98de5f-b1cd-4ce8-8bc6-61b11f0bd22e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caa1ecca8cfb45ea80d6bbf6db8332a7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:17.123588 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-aa98de5f-b1cd-4ce8-8bc6-61b11f0bd22e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caa1ecca8cfb45ea80d6bbf6db8332a7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:17.124322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-aa98de5f-b1cd-4ce8-8bc6-61b11f0bd22e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caa1ecca8cfb45ea80d6bbf6db8332a7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:17.192125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-aa98de5f-b1cd-4ce8-8bc6-61b11f0bd22e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caa1ecca8cfb45ea80d6bbf6db8332a7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:17.192778 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-aa98de5f-b1cd-4ce8-8bc6-61b11f0bd22e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caa1ecca8cfb45ea80d6bbf6db8332a7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:17.206777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1467/2901] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:27:16 2022] DELETE /identity/v3/users/caa1ecca8cfb45ea80d6bbf6db8332a7 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:27:17.633647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70c20d-555f-4c13-8306-c16f10a9bd12 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:17.636847 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0b70c20d-555f-4c13-8306-c16f10a9bd12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:17.652649 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70c20d-555f-4c13-8306-c16f10a9bd12 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:17.711870 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ab844-9ba9-4136-a500-6d850d3d3c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:17.712123 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ab844-9ba9-4136-a500-6d850d3d3c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:17.712523 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ab844-9ba9-4136-a500-6d850d3d3c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:17.712801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ab844-9ba9-4136-a500-6d850d3d3c8f admin admin] PATH_INFO: `/v3/projects/8ea3c1ea759846bea30df1b9a5f49214` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:17.713959 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ab844-9ba9-4136-a500-6d850d3d3c8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ea3c1ea759846bea30df1b9a5f49214)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:17.716037 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ab844-9ba9-4136-a500-6d850d3d3c8f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:17.836395 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:17.839472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-dd9ab844-9ba9-4136-a500-6d850d3d3c8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ea3c1ea759846bea30df1b9a5f49214', 'request_id': 'req-dd9ab844-9ba9-4136-a500-6d850d3d3c8f'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:17.845985 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-dd9ab844-9ba9-4136-a500-6d850d3d3c8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ea3c1ea759846bea30df1b9a5f49214', 'request_id': 'req-dd9ab844-9ba9-4136-a500-6d850d3d3c8f'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:17.908982 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-dd9ab844-9ba9-4136-a500-6d850d3d3c8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ea3c1ea759846bea30df1b9a5f49214', 'request_id': 'req-dd9ab844-9ba9-4136-a500-6d850d3d3c8f'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:17.922487 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-dd9ab844-9ba9-4136-a500-6d850d3d3c8f admin admin] The token cache is being invalidate because project 8ea3c1ea759846bea30df1b9a5f49214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:27:17.922775 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-dd9ab844-9ba9-4136-a500-6d850d3d3c8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:17.923496 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-dd9ab844-9ba9-4136-a500-6d850d3d3c8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:27:17.925053 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1435/2902] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:27:17 2022] DELETE /identity/v3/projects/8ea3c1ea759846bea30df1b9a5f49214 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:27:17.930040 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390c102-7115-42e7-8526-bb6b144983fd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:17.930523 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390c102-7115-42e7-8526-bb6b144983fd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:17.930740 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390c102-7115-42e7-8526-bb6b144983fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:17.952434 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9390c102-7115-42e7-8526-bb6b144983fd None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:27:17.953649 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9390c102-7115-42e7-8526-bb6b144983fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:17.993968 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:18.018556 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1468/2903] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:27:18.026478 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f463ac-32ee-4876-9692-e85fec8431aa None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:18.026760 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f463ac-32ee-4876-9692-e85fec8431aa None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:18.027070 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f463ac-32ee-4876-9692-e85fec8431aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:18.047498 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-f5f463ac-32ee-4876-9692-e85fec8431aa None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:27:18.048716 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-f5f463ac-32ee-4876-9692-e85fec8431aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:18.057880 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1436/2904] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:27:18.063449 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390c102-7115-42e7-8526-bb6b144983fd None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:18.073172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390c102-7115-42e7-8526-bb6b144983fd None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:18.078459 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1e16c-9788-4005-94d4-e1ef4e85c614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:18.079110 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1e16c-9788-4005-94d4-e1ef4e85c614 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:18.079537 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1e16c-9788-4005-94d4-e1ef4e85c614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:18.080172 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1e16c-9788-4005-94d4-e1ef4e85c614 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:18.081058 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1e16c-9788-4005-94d4-e1ef4e85c614 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 06 08:27:18.081367 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1e16c-9788-4005-94d4-e1ef4e85c614 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:18.083347 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1e16c-9788-4005-94d4-e1ef4e85c614 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:18.095679 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1469/2905] 10.222.0.30 () {58 vars in 1153 bytes} [Thu Oct 6 08:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 06 08:27:18.100226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f463ac-32ee-4876-9692-e85fec8431aa None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:18.110154 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f463ac-32ee-4876-9692-e85fec8431aa None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:18.114761 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3ff4e-c073-4536-994d-f3642409456a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:18.115382 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3ff4e-c073-4536-994d-f3642409456a admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:18.115656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3ff4e-c073-4536-994d-f3642409456a admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:18.115918 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3ff4e-c073-4536-994d-f3642409456a admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:18.116564 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3ff4e-c073-4536-994d-f3642409456a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:18.118464 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3ff4e-c073-4536-994d-f3642409456a admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:18.208414 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1437/2906] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:27:18 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:27:18.212780 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1e16c-9788-4005-94d4-e1ef4e85c614 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:18.224247 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1e16c-9788-4005-94d4-e1ef4e85c614 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:18.282559 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0b537-0e80-484c-b693-464695a5f99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:18.283217 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0b537-0e80-484c-b693-464695a5f99a admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:18.283523 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0b537-0e80-484c-b693-464695a5f99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:18.283806 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0b537-0e80-484c-b693-464695a5f99a admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:18.284435 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0b537-0e80-484c-b693-464695a5f99a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:18.286374 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0b537-0e80-484c-b693-464695a5f99a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:18.381283 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1470/2907] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:27:18 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 06 08:27:18.385537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3ff4e-c073-4536-994d-f3642409456a admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:18.396796 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3ff4e-c073-4536-994d-f3642409456a admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:18.401449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7d78e-9f60-4267-8209-285a0308bec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:18.402038 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7d78e-9f60-4267-8209-285a0308bec2 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:18.402312 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7d78e-9f60-4267-8209-285a0308bec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:18.402582 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7d78e-9f60-4267-8209-285a0308bec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:18.403254 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7d78e-9f60-4267-8209-285a0308bec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:18.405263 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7d78e-9f60-4267-8209-285a0308bec2 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:18.414869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1438/2908] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:27:18.418787 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0b537-0e80-484c-b693-464695a5f99a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:18.429625 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0b537-0e80-484c-b693-464695a5f99a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:18.434281 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03ae82-521e-4621-ab55-1457be347537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:18.434896 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03ae82-521e-4621-ab55-1457be347537 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:18.435209 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03ae82-521e-4621-ab55-1457be347537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:18.436092 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03ae82-521e-4621-ab55-1457be347537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:18.436850 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03ae82-521e-4621-ab55-1457be347537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:18.438427 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03ae82-521e-4621-ab55-1457be347537 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:18.448899 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1471/2909] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:27:18.454537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7d78e-9f60-4267-8209-285a0308bec2 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:18.464353 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7d78e-9f60-4267-8209-285a0308bec2 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:18.469564 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f14620-2980-4144-ad25-8fcbb32dce7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:18.470226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f14620-2980-4144-ad25-8fcbb32dce7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:18.470504 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f14620-2980-4144-ad25-8fcbb32dce7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:18.470777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f14620-2980-4144-ad25-8fcbb32dce7f admin admin] PATH_INFO: `/v3/projects/7cc3f986a0af4426b9995cdcaa66c1b0/users/3d6e390b824c41bc91fbd83f922cae97/roles/dd5cafa908a74cfd8e1a8baa44122242` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:18.485961 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f14620-2980-4144-ad25-8fcbb32dce7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cc3f986a0af4426b9995cdcaa66c1b0, user_id=3d6e390b824c41bc91fbd83f922cae97, role_id=dd5cafa908a74cfd8e1a8baa44122242)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:18.488656 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f14620-2980-4144-ad25-8fcbb32dce7f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:18.560537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1439/2910] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:27:18 2022] PUT /identity/v3/projects/7cc3f986a0af4426b9995cdcaa66c1b0/users/3d6e390b824c41bc91fbd83f922cae97/roles/dd5cafa908a74cfd8e1a8baa44122242 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:27:18.565915 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b9abe-da7c-400a-b0ff-7cd5607fd547 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:18.566197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b9abe-da7c-400a-b0ff-7cd5607fd547 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:18.566472 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b9abe-da7c-400a-b0ff-7cd5607fd547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:18.585345 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f48b9abe-da7c-400a-b0ff-7cd5607fd547 None None] MFA Rules not processed for user `93747f497a1f4adcb4b8ddb03adfe01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:27:18.586556 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f48b9abe-da7c-400a-b0ff-7cd5607fd547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:18.645214 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1472/2911] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:27:18.655666 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f14620-2980-4144-ad25-8fcbb32dce7f admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:18.657321 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-28f14620-2980-4144-ad25-8fcbb32dce7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:18.713606 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f14620-2980-4144-ad25-8fcbb32dce7f admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:18.717845 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6ff30-ccbd-4d2b-8c7b-582a98d28347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:18.718453 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6ff30-ccbd-4d2b-8c7b-582a98d28347 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:18.718732 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6ff30-ccbd-4d2b-8c7b-582a98d28347 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:18.719039 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6ff30-ccbd-4d2b-8c7b-582a98d28347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:18.728169 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6ff30-ccbd-4d2b-8c7b-582a98d28347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:18.729617 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6ff30-ccbd-4d2b-8c7b-582a98d28347 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:18.739651 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1440/2912] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:27:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:27:23.958591 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d347e1b-7f84-442d-ac61-fa3351e287e0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:23.959146 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d347e1b-7f84-442d-ac61-fa3351e287e0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:23.959238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d347e1b-7f84-442d-ac61-fa3351e287e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:23.992590 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9d347e1b-7f84-442d-ac61-fa3351e287e0 None None] MFA Rules not processed for user `3d6e390b824c41bc91fbd83f922cae97`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:27:23.993877 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9d347e1b-7f84-442d-ac61-fa3351e287e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:24.034922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:24.062401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1473/2913] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:27:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:27:24.067067 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6ff30-ccbd-4d2b-8c7b-582a98d28347 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:24.077257 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6ff30-ccbd-4d2b-8c7b-582a98d28347 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:24.081968 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2f80c-3c53-4c0e-80e8-8aa6f9036ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:24.082577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2f80c-3c53-4c0e-80e8-8aa6f9036ad7 admin admin] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:24.083108 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2f80c-3c53-4c0e-80e8-8aa6f9036ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:24.083437 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2f80c-3c53-4c0e-80e8-8aa6f9036ad7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:24.084188 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2f80c-3c53-4c0e-80e8-8aa6f9036ad7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:24.086123 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2f80c-3c53-4c0e-80e8-8aa6f9036ad7 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:24.187477 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1441/2914] 10.222.0.30 () {60 vars in 1153 bytes} [Thu Oct 6 08:27:24 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:27:24.191579 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d347e1b-7f84-442d-ac61-fa3351e287e0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:24.205189 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d347e1b-7f84-442d-ac61-fa3351e287e0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:24.269315 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50e850-0f57-4999-b70f-ed1e21ec6649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:24.269919 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50e850-0f57-4999-b70f-ed1e21ec6649 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:24.270181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50e850-0f57-4999-b70f-ed1e21ec6649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:24.270479 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50e850-0f57-4999-b70f-ed1e21ec6649 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:24.271163 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50e850-0f57-4999-b70f-ed1e21ec6649 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:24.273081 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50e850-0f57-4999-b70f-ed1e21ec6649 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:24.381795 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1474/2915] 10.222.0.30 () {60 vars in 1144 bytes} [Thu Oct 6 08:27:24 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 06 08:27:24.385338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2f80c-3c53-4c0e-80e8-8aa6f9036ad7 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:24.401110 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2f80c-3c53-4c0e-80e8-8aa6f9036ad7 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:24.407159 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d48de5-eaf0-4850-880a-0b4bdd15f809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:24.407831 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d48de5-eaf0-4850-880a-0b4bdd15f809 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:24.408293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d48de5-eaf0-4850-880a-0b4bdd15f809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:24.408568 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d48de5-eaf0-4850-880a-0b4bdd15f809 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:24.409281 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d48de5-eaf0-4850-880a-0b4bdd15f809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:24.410934 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d48de5-eaf0-4850-880a-0b4bdd15f809 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:24.424704 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1442/2916] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 06 08:27:24.429131 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50e850-0f57-4999-b70f-ed1e21ec6649 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:24.440829 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50e850-0f57-4999-b70f-ed1e21ec6649 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:24.446125 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22e166-8ef2-422a-a230-659245e4f5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:24.446862 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22e166-8ef2-422a-a230-659245e4f5f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:24.447238 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22e166-8ef2-422a-a230-659245e4f5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:24.447515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22e166-8ef2-422a-a230-659245e4f5f5 admin admin] PATH_INFO: `/v3/projects/91eef916bfda4975a8b3d32ad2cc2989/users/31c09699a275477e8e6c1d575cfa7c94/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:24.463024 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22e166-8ef2-422a-a230-659245e4f5f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91eef916bfda4975a8b3d32ad2cc2989, user_id=31c09699a275477e8e6c1d575cfa7c94, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:24.465919 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22e166-8ef2-422a-a230-659245e4f5f5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:24.563844 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1475/2917] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:27:24 2022] PUT /identity/v3/projects/91eef916bfda4975a8b3d32ad2cc2989/users/31c09699a275477e8e6c1d575cfa7c94/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:27:24.567712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d48de5-eaf0-4850-880a-0b4bdd15f809 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:24.578128 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d48de5-eaf0-4850-880a-0b4bdd15f809 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:24.627371 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:24.647115 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b2e44-51d5-4c5c-846d-6da1677f6a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:24.647817 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b2e44-51d5-4c5c-846d-6da1677f6a91 admin admin] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:24.648029 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b2e44-51d5-4c5c-846d-6da1677f6a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:24.652009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b2e44-51d5-4c5c-846d-6da1677f6a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:24.652009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b2e44-51d5-4c5c-846d-6da1677f6a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:24.652009 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b2e44-51d5-4c5c-846d-6da1677f6a91 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:24.661457 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1443/2918] 10.222.0.30 () {58 vars in 1122 bytes} [Thu Oct 6 08:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 06 08:27:24.665553 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22e166-8ef2-422a-a230-659245e4f5f5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:24.676832 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22e166-8ef2-422a-a230-659245e4f5f5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:24.681429 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e4fcf-6bc6-473e-a0c7-2ac022c0de4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:24.682064 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e4fcf-6bc6-473e-a0c7-2ac022c0de4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:24.682340 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e4fcf-6bc6-473e-a0c7-2ac022c0de4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:24.682608 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e4fcf-6bc6-473e-a0c7-2ac022c0de4f admin admin] PATH_INFO: `/v3/projects/91eef916bfda4975a8b3d32ad2cc2989/users/31c09699a275477e8e6c1d575cfa7c94/roles/4c34f0f69c3a408fa32dbc52c64e7cfb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:24.684587 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e4fcf-6bc6-473e-a0c7-2ac022c0de4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=91eef916bfda4975a8b3d32ad2cc2989, user_id=31c09699a275477e8e6c1d575cfa7c94, role_id=4c34f0f69c3a408fa32dbc52c64e7cfb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:24.687530 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e4fcf-6bc6-473e-a0c7-2ac022c0de4f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:24.698655 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1476/2919] 10.222.0.30 () {60 vars in 1483 bytes} [Thu Oct 6 08:27:24 2022] PUT /identity/v3/projects/91eef916bfda4975a8b3d32ad2cc2989/users/31c09699a275477e8e6c1d575cfa7c94/roles/4c34f0f69c3a408fa32dbc52c64e7cfb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:27:30.303245 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab093df2-dd60-4c05-ad1b-6c167036225e None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:30.304044 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab093df2-dd60-4c05-ad1b-6c167036225e None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:30.304326 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab093df2-dd60-4c05-ad1b-6c167036225e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:30.340226 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-ab093df2-dd60-4c05-ad1b-6c167036225e None None] MFA Rules not processed for user `31c09699a275477e8e6c1d575cfa7c94`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:27:30.341429 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-ab093df2-dd60-4c05-ad1b-6c167036225e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:30.432789 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1444/2920] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:27:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:27:30.441566 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e4fcf-6bc6-473e-a0c7-2ac022c0de4f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:30.443860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-628e4fcf-6bc6-473e-a0c7-2ac022c0de4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:30.531967 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e4fcf-6bc6-473e-a0c7-2ac022c0de4f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:30.538415 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadea840-19b4-4dc0-9612-f4aec3da311e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:30.539069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadea840-19b4-4dc0-9612-f4aec3da311e service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:30.539349 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadea840-19b4-4dc0-9612-f4aec3da311e service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:30.539620 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadea840-19b4-4dc0-9612-f4aec3da311e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:30.569115 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:30.601612 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadea840-19b4-4dc0-9612-f4aec3da311e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:30.603305 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadea840-19b4-4dc0-9612-f4aec3da311e service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:30.613181 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1477/2921] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:27:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:27:33.328835 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83853ddd-a564-4b82-84ca-3c6c7f210ad9 None None] REQUEST_METHOD: `POST` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:33.329333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83853ddd-a564-4b82-84ca-3c6c7f210ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:33.329447 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83853ddd-a564-4b82-84ca-3c6c7f210ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:33.350238 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.auth.core [None req-83853ddd-a564-4b82-84ca-3c6c7f210ad9 None None] MFA Rules not processed for user `3d6e390b824c41bc91fbd83f922cae97`. Rule list: `[]` (Enabled: `True`). {{(pid=81479) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:27:33.351471 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-83853ddd-a564-4b82-84ca-3c6c7f210ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:33.360639 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1445/2922] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:27:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:27:40.849826 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadea840-19b4-4dc0-9612-f4aec3da311e service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:40.851409 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fadea840-19b4-4dc0-9612-f4aec3da311e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:40.928392 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadea840-19b4-4dc0-9612-f4aec3da311e service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:40.933653 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36bc52-feda-48ee-825d-fb21f0e9c1c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6074872303d3468fb566378fa34e8bea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:40.934266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36bc52-feda-48ee-825d-fb21f0e9c1c8 service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:40.934543 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36bc52-feda-48ee-825d-fb21f0e9c1c8 service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:40.934811 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36bc52-feda-48ee-825d-fb21f0e9c1c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:40.936606 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fa36bc52-feda-48ee-825d-fb21f0e9c1c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:40.952563 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36bc52-feda-48ee-825d-fb21f0e9c1c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:40.954441 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36bc52-feda-48ee-825d-fb21f0e9c1c8 service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:40.965451 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1478/2923] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:27:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:27:43.344481 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=48 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:50.960672 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=47 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:27:55.504751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83853ddd-a564-4b82-84ca-3c6c7f210ad9 None None] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:27:55.510917 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-83853ddd-a564-4b82-84ca-3c6c7f210ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:55.562647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83853ddd-a564-4b82-84ca-3c6c7f210ad9 None None] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:27:55.566804 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd4548-19a3-4d2e-b919-925de41aa722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:27:55.567432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd4548-19a3-4d2e-b919-925de41aa722 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:27:55.567710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd4548-19a3-4d2e-b919-925de41aa722 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:27:55.567979 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd4548-19a3-4d2e-b919-925de41aa722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:27:55.569685 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-7efd4548-19a3-4d2e-b919-925de41aa722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:27:55.579990 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd4548-19a3-4d2e-b919-925de41aa722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:27:55.581421 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd4548-19a3-4d2e-b919-925de41aa722 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:27:55.589333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1446/2924] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:27:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:28:05.585801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=26 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:28:18.398698 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36bc52-feda-48ee-825d-fb21f0e9c1c8 service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:28:18.408705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36bc52-feda-48ee-825d-fb21f0e9c1c8 service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:28:18.412328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6c854-dec1-45e0-a8eb-d4379ff64fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:28:18.412922 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6c854-dec1-45e0-a8eb-d4379ff64fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:28:18.413197 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6c854-dec1-45e0-a8eb-d4379ff64fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:28:18.413465 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6c854-dec1-45e0-a8eb-d4379ff64fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:28:18.422964 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d6c854-dec1-45e0-a8eb-d4379ff64fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:28:18.424391 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d6c854-dec1-45e0-a8eb-d4379ff64fb1 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:28:18.433954 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1479/2925] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:28:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:28:28.429596 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:29:38.519190 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd4548-19a3-4d2e-b919-925de41aa722 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:29:38.527980 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd4548-19a3-4d2e-b919-925de41aa722 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:29:38.532132 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf7102-9e24-4de9-8503-d383b7d4483f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:29:38.532733 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf7102-9e24-4de9-8503-d383b7d4483f service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:29:38.533008 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf7102-9e24-4de9-8503-d383b7d4483f service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:29:38.533274 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf7102-9e24-4de9-8503-d383b7d4483f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:29:38.535043 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-0fcf7102-9e24-4de9-8503-d383b7d4483f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:29:38.545057 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf7102-9e24-4de9-8503-d383b7d4483f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:29:38.546485 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf7102-9e24-4de9-8503-d383b7d4483f service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:29:38.554434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1447/2926] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:29:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:29:41.033721 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6c854-dec1-45e0-a8eb-d4379ff64fb1 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:29:41.042746 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6c854-dec1-45e0-a8eb-d4379ff64fb1 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:29:41.060174 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e42d04-2e80-483b-9826-f702130ccda1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:29:41.060174 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e42d04-2e80-483b-9826-f702130ccda1 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:29:41.060174 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e42d04-2e80-483b-9826-f702130ccda1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:29:41.060174 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e42d04-2e80-483b-9826-f702130ccda1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:29:41.060632 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-51e42d04-2e80-483b-9826-f702130ccda1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:29:41.065921 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e42d04-2e80-483b-9826-f702130ccda1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:29:41.067440 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e42d04-2e80-483b-9826-f702130ccda1 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:29:41.079247 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1480/2927] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:29:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:29:42.273665 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf7102-9e24-4de9-8503-d383b7d4483f service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:29:42.282822 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf7102-9e24-4de9-8503-d383b7d4483f service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:29:42.286338 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c5966-2bfd-4fa5-918a-ff4ae436f50d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:29:42.286919 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c5966-2bfd-4fa5-918a-ff4ae436f50d service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:29:42.287224 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c5966-2bfd-4fa5-918a-ff4ae436f50d service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:29:42.287489 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c5966-2bfd-4fa5-918a-ff4ae436f50d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:29:42.289183 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-746c5966-2bfd-4fa5-918a-ff4ae436f50d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:29:42.302889 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c5966-2bfd-4fa5-918a-ff4ae436f50d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:29:42.304333 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c5966-2bfd-4fa5-918a-ff4ae436f50d service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:29:42.315345 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1448/2928] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:29:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:29:45.778515 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff669e4c-6391-4fd7-8983-ec9d85bfa1b3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:29:45.779012 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff669e4c-6391-4fd7-8983-ec9d85bfa1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:29:45.779132 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff669e4c-6391-4fd7-8983-ec9d85bfa1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:29:45.798870 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ff669e4c-6391-4fd7-8983-ec9d85bfa1b3 None None] MFA Rules not processed for user `79d29aa9482f4c1f9b1554333d85d80a`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:29:45.800082 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff669e4c-6391-4fd7-8983-ec9d85bfa1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:29:45.855469 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1481/2929] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:29:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:29:52.310178 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=12 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:29:55.847718 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=31 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:30:06.585380 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c5966-2bfd-4fa5-918a-ff4ae436f50d service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:30:06.587034 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-746c5966-2bfd-4fa5-918a-ff4ae436f50d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:30:06.654175 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c5966-2bfd-4fa5-918a-ff4ae436f50d service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:30:06.659946 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bce68-9b96-4189-ab6b-03c84ee6eb2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:30:06.660547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bce68-9b96-4189-ab6b-03c84ee6eb2c service nova] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:30:06.660828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bce68-9b96-4189-ab6b-03c84ee6eb2c service nova] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:30:06.661126 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bce68-9b96-4189-ab6b-03c84ee6eb2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:30:06.662869 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-159bce68-9b96-4189-ab6b-03c84ee6eb2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:30:06.676193 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bce68-9b96-4189-ab6b-03c84ee6eb2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:30:06.680319 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bce68-9b96-4189-ab6b-03c84ee6eb2c service nova] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:30:06.688630 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1449/2930] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:30:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:30:16.683944 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=28 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:30:49.463007 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff669e4c-6391-4fd7-8983-ec9d85bfa1b3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:30:49.486196 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff669e4c-6391-4fd7-8983-ec9d85bfa1b3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:30:49.490265 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605826d0-2b1b-4a6d-b98c-eb6155bde7f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:30:49.490860 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605826d0-2b1b-4a6d-b98c-eb6155bde7f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:30:49.491166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605826d0-2b1b-4a6d-b98c-eb6155bde7f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:30:49.491437 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605826d0-2b1b-4a6d-b98c-eb6155bde7f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:30:49.493164 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-605826d0-2b1b-4a6d-b98c-eb6155bde7f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:30:49.516632 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605826d0-2b1b-4a6d-b98c-eb6155bde7f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:30:49.518881 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605826d0-2b1b-4a6d-b98c-eb6155bde7f3 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:30:49.527705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1482/2931] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:30:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:30:50.904019 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bce68-9b96-4189-ab6b-03c84ee6eb2c service nova] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:30:50.914293 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bce68-9b96-4189-ab6b-03c84ee6eb2c service nova] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:30:50.918260 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bf480-4fe2-480f-a98c-8c2a76bcb62d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:30:50.919273 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bf480-4fe2-480f-a98c-8c2a76bcb62d service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:30:50.919550 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bf480-4fe2-480f-a98c-8c2a76bcb62d service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:30:50.920040 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bf480-4fe2-480f-a98c-8c2a76bcb62d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:30:50.921795 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1a9bf480-4fe2-480f-a98c-8c2a76bcb62d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:30:50.932569 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bf480-4fe2-480f-a98c-8c2a76bcb62d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:30:50.934031 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bf480-4fe2-480f-a98c-8c2a76bcb62d service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:30:50.943232 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1450/2932] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:30:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:30:51.025568 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605826d0-2b1b-4a6d-b98c-eb6155bde7f3 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:30:51.035166 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605826d0-2b1b-4a6d-b98c-eb6155bde7f3 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:30:51.038717 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa640d-7221-4dbc-ab68-3b3e0d2d3348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:30:51.039526 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa640d-7221-4dbc-ab68-3b3e0d2d3348 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:30:51.039800 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa640d-7221-4dbc-ab68-3b3e0d2d3348 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:30:51.040069 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa640d-7221-4dbc-ab68-3b3e0d2d3348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:30:51.041848 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1daa640d-7221-4dbc-ab68-3b3e0d2d3348 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:30:51.055350 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa640d-7221-4dbc-ab68-3b3e0d2d3348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:30:51.056825 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa640d-7221-4dbc-ab68-3b3e0d2d3348 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:30:51.067887 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1483/2933] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:30:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:31:00.938536 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=6 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:31:01.062799 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=23 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:31:55.468227 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bf480-4fe2-480f-a98c-8c2a76bcb62d service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:31:55.470205 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-1a9bf480-4fe2-480f-a98c-8c2a76bcb62d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:31:55.548922 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bf480-4fe2-480f-a98c-8c2a76bcb62d service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:31:55.552608 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c6b0e-e928-4335-b857-1179041776f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a899c6dabca948269f6c1370425f244b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:31:55.553200 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c6b0e-e928-4335-b857-1179041776f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:31:55.553475 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c6b0e-e928-4335-b857-1179041776f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:31:55.553743 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c6b0e-e928-4335-b857-1179041776f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:31:55.555469 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5e5c6b0e-e928-4335-b857-1179041776f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:31:55.567103 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c6b0e-e928-4335-b857-1179041776f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:31:55.568521 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c6b0e-e928-4335-b857-1179041776f7 service cinder] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:31:55.577276 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1451/2934] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:31:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:32:05.573223 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=35 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:32:07.320807 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb053277-e0fe-4db0-85e7-51794a1c8538 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:32:07.321344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb053277-e0fe-4db0-85e7-51794a1c8538 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:32:07.321344 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb053277-e0fe-4db0-85e7-51794a1c8538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:32:07.340221 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-eb053277-e0fe-4db0-85e7-51794a1c8538 None None] MFA Rules not processed for user `79d29aa9482f4c1f9b1554333d85d80a`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 06 08:32:07.341402 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-eb053277-e0fe-4db0-85e7-51794a1c8538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:32:07.348815 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1484/2935] 10.222.0.30 () {58 vars in 958 bytes} [Thu Oct 6 08:32:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 06 08:32:08.345861 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c6b0e-e928-4335-b857-1179041776f7 service cinder] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:32:08.347420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-5e5c6b0e-e928-4335-b857-1179041776f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:32:08.363045 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c6b0e-e928-4335-b857-1179041776f7 service cinder] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:32:08.368512 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff28ef4-0d0a-4362-8bb2-c8382ca99066 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6074872303d3468fb566378fa34e8bea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:32:08.369307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff28ef4-0d0a-4362-8bb2-c8382ca99066 service placement] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:32:08.369453 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff28ef4-0d0a-4362-8bb2-c8382ca99066 service placement] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:32:08.369831 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff28ef4-0d0a-4362-8bb2-c8382ca99066 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:32:08.371732 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-9ff28ef4-0d0a-4362-8bb2-c8382ca99066 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:32:08.384824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff28ef4-0d0a-4362-8bb2-c8382ca99066 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:32:08.386420 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff28ef4-0d0a-4362-8bb2-c8382ca99066 service placement] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:32:08.403373 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1452/2936] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:32:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 06 08:32:17.335074 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:32:18.392808 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=6 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:32:24.192805 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb053277-e0fe-4db0-85e7-51794a1c8538 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:32:24.213624 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb053277-e0fe-4db0-85e7-51794a1c8538 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:32:24.219657 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0f656-eeaa-40f5-8479-92716a5a219b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:32:24.220422 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0f656-eeaa-40f5-8479-92716a5a219b service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:32:24.220742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0f656-eeaa-40f5-8479-92716a5a219b service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:32:24.221014 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0f656-eeaa-40f5-8479-92716a5a219b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:32:24.222784 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2ce0f656-eeaa-40f5-8479-92716a5a219b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:32:24.235674 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0f656-eeaa-40f5-8479-92716a5a219b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:32:24.237824 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0f656-eeaa-40f5-8479-92716a5a219b service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:32:24.248464 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1485/2937] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:32:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:32:27.325028 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff28ef4-0d0a-4362-8bb2-c8382ca99066 service placement] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:32:27.335149 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff28ef4-0d0a-4362-8bb2-c8382ca99066 service placement] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:32:27.338674 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e49e1-45d6-4032-b84f-78133f2dafe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:32:27.339301 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e49e1-45d6-4032-b84f-78133f2dafe0 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:32:27.339580 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e49e1-45d6-4032-b84f-78133f2dafe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:32:27.339848 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e49e1-45d6-4032-b84f-78133f2dafe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:32:27.341558 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-b26e49e1-45d6-4032-b84f-78133f2dafe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:32:27.403909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e49e1-45d6-4032-b84f-78133f2dafe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:32:27.405428 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e49e1-45d6-4032-b84f-78133f2dafe0 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:32:27.415062 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1453/2938] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:32:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:32:34.244165 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=9 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:32:35.496000 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0f656-eeaa-40f5-8479-92716a5a219b service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:32:35.506275 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0f656-eeaa-40f5-8479-92716a5a219b service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:32:35.510005 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dab4b-8c14-4baa-8e12-fab6ea353787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:32:35.510591 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dab4b-8c14-4baa-8e12-fab6ea353787 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:32:35.510863 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dab4b-8c14-4baa-8e12-fab6ea353787 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:32:35.511159 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dab4b-8c14-4baa-8e12-fab6ea353787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:32:35.520038 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42dab4b-8c14-4baa-8e12-fab6ea353787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:32:35.521518 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42dab4b-8c14-4baa-8e12-fab6ea353787 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:32:35.530266 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1486/2939] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:32:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:32:37.410752 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=30 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:32:40.238357 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e49e1-45d6-4032-b84f-78133f2dafe0 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:32:40.259378 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e49e1-45d6-4032-b84f-78133f2dafe0 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:32:40.264647 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434ffa9-b30f-461f-afc2-953527f775d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6074872303d3468fb566378fa34e8bea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:32:40.265249 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434ffa9-b30f-461f-afc2-953527f775d3 service placement] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:32:40.265527 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434ffa9-b30f-461f-afc2-953527f775d3 service placement] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:32:40.265801 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434ffa9-b30f-461f-afc2-953527f775d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:32:40.275555 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434ffa9-b30f-461f-afc2-953527f775d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:32:40.277152 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434ffa9-b30f-461f-afc2-953527f775d3 service placement] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:32:40.289472 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1454/2940] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:32:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:32:42.772207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dab4b-8c14-4baa-8e12-fab6ea353787 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:32:42.781029 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dab4b-8c14-4baa-8e12-fab6ea353787 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:32:42.784725 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1bfb0-9678-4ffc-aad4-94d5b037ae6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:32:42.785787 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1bfb0-9678-4ffc-aad4-94d5b037ae6e service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:32:42.786063 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1bfb0-9678-4ffc-aad4-94d5b037ae6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:32:42.786328 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1bfb0-9678-4ffc-aad4-94d5b037ae6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:32:42.857077 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1bfb0-9678-4ffc-aad4-94d5b037ae6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:32:42.858432 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1bfb0-9678-4ffc-aad4-94d5b037ae6e service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:32:42.877207 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1487/2941] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:32:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:32:46.432792 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434ffa9-b30f-461f-afc2-953527f775d3 service placement] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:32:46.446449 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434ffa9-b30f-461f-afc2-953527f775d3 service placement] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:32:46.452434 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22031d4e-d37a-4015-b3f8-6878cc382bae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6074872303d3468fb566378fa34e8bea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:32:46.453045 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22031d4e-d37a-4015-b3f8-6878cc382bae service placement] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:32:46.453288 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22031d4e-d37a-4015-b3f8-6878cc382bae service placement] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:32:46.453587 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22031d4e-d37a-4015-b3f8-6878cc382bae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:32:46.455400 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-22031d4e-d37a-4015-b3f8-6878cc382bae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:32:46.483250 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22031d4e-d37a-4015-b3f8-6878cc382bae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:32:46.484883 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22031d4e-d37a-4015-b3f8-6878cc382bae service placement] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:32:46.496130 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1455/2942] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:32:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:32:50.303397 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1bfb0-9678-4ffc-aad4-94d5b037ae6e service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:32:50.313337 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1bfb0-9678-4ffc-aad4-94d5b037ae6e service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:32:50.321322 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad913abd-9b1f-4537-8857-f264a6acfe61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:32:50.321951 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad913abd-9b1f-4537-8857-f264a6acfe61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:32:50.322227 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad913abd-9b1f-4537-8857-f264a6acfe61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:32:50.322496 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad913abd-9b1f-4537-8857-f264a6acfe61 admin admin] PATH_INFO: `/v3/users/3d6e390b824c41bc91fbd83f922cae97` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:32:50.323689 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad913abd-9b1f-4537-8857-f264a6acfe61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d6e390b824c41bc91fbd83f922cae97)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:32:50.330794 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad913abd-9b1f-4537-8857-f264a6acfe61 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:32:50.500389 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ad913abd-9b1f-4537-8857-f264a6acfe61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d6e390b824c41bc91fbd83f922cae97'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:50.512709 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ad913abd-9b1f-4537-8857-f264a6acfe61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d6e390b824c41bc91fbd83f922cae97'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:50.513390 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ad913abd-9b1f-4537-8857-f264a6acfe61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d6e390b824c41bc91fbd83f922cae97'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:50.568366 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ad913abd-9b1f-4537-8857-f264a6acfe61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d6e390b824c41bc91fbd83f922cae97'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:50.568984 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ad913abd-9b1f-4537-8857-f264a6acfe61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d6e390b824c41bc91fbd83f922cae97'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:50.581599 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1488/2943] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:32:50 2022] DELETE /identity/v3/users/3d6e390b824c41bc91fbd83f922cae97 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:32:50.833946 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22031d4e-d37a-4015-b3f8-6878cc382bae service placement] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:32:50.844448 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22031d4e-d37a-4015-b3f8-6878cc382bae service placement] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:32:50.903180 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22142b5-f235-4f5f-b217-01fd34e45285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:32:50.903881 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22142b5-f235-4f5f-b217-01fd34e45285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:32:50.904165 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22142b5-f235-4f5f-b217-01fd34e45285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:32:50.904600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22142b5-f235-4f5f-b217-01fd34e45285 admin admin] PATH_INFO: `/v3/projects/7cc3f986a0af4426b9995cdcaa66c1b0` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:32:50.905780 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22142b5-f235-4f5f-b217-01fd34e45285 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cc3f986a0af4426b9995cdcaa66c1b0)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:32:50.907751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22142b5-f235-4f5f-b217-01fd34e45285 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:32:51.031876 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d22142b5-f235-4f5f-b217-01fd34e45285 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cc3f986a0af4426b9995cdcaa66c1b0', 'request_id': 'req-d22142b5-f235-4f5f-b217-01fd34e45285'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:51.038755 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d22142b5-f235-4f5f-b217-01fd34e45285 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cc3f986a0af4426b9995cdcaa66c1b0', 'request_id': 'req-d22142b5-f235-4f5f-b217-01fd34e45285'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:51.104900 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d22142b5-f235-4f5f-b217-01fd34e45285 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cc3f986a0af4426b9995cdcaa66c1b0', 'request_id': 'req-d22142b5-f235-4f5f-b217-01fd34e45285'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:51.127803 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d22142b5-f235-4f5f-b217-01fd34e45285 admin admin] The token cache is being invalidate because project 7cc3f986a0af4426b9995cdcaa66c1b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:32:51.128156 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d22142b5-f235-4f5f-b217-01fd34e45285 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:51.132899 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-d22142b5-f235-4f5f-b217-01fd34e45285 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:51.132899 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1456/2944] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:32:50 2022] DELETE /identity/v3/projects/7cc3f986a0af4426b9995cdcaa66c1b0 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:32:51.134371 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad913abd-9b1f-4537-8857-f264a6acfe61 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:32:51.135817 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ad913abd-9b1f-4537-8857-f264a6acfe61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:32:51.152447 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad913abd-9b1f-4537-8857-f264a6acfe61 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:32:51.185078 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:32:51.209336 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa975b7-0cf1-4a16-b08f-f66f838f5ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:32:51.209958 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa975b7-0cf1-4a16-b08f-f66f838f5ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:32:51.210236 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa975b7-0cf1-4a16-b08f-f66f838f5ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:32:51.210509 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa975b7-0cf1-4a16-b08f-f66f838f5ecd admin admin] PATH_INFO: `/v3/users/31c09699a275477e8e6c1d575cfa7c94` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:32:51.211676 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa975b7-0cf1-4a16-b08f-f66f838f5ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c09699a275477e8e6c1d575cfa7c94)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:32:51.213627 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa975b7-0cf1-4a16-b08f-f66f838f5ecd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:32:51.354026 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5fa975b7-0cf1-4a16-b08f-f66f838f5ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c09699a275477e8e6c1d575cfa7c94'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:51.366901 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5fa975b7-0cf1-4a16-b08f-f66f838f5ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c09699a275477e8e6c1d575cfa7c94'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:51.367607 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5fa975b7-0cf1-4a16-b08f-f66f838f5ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c09699a275477e8e6c1d575cfa7c94'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:51.432762 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5fa975b7-0cf1-4a16-b08f-f66f838f5ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c09699a275477e8e6c1d575cfa7c94'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:51.433380 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5fa975b7-0cf1-4a16-b08f-f66f838f5ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c09699a275477e8e6c1d575cfa7c94'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:51.446262 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1489/2945] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:32:51 2022] DELETE /identity/v3/users/31c09699a275477e8e6c1d575cfa7c94 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:32:51.675694 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22142b5-f235-4f5f-b217-01fd34e45285 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:32:51.677279 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-d22142b5-f235-4f5f-b217-01fd34e45285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:32:51.690553 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22142b5-f235-4f5f-b217-01fd34e45285 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:32:51.723307 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:32:51.749742 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd5041-5c9f-463a-bd8d-6756e488e7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:32:51.750366 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd5041-5c9f-463a-bd8d-6756e488e7fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:32:51.750660 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd5041-5c9f-463a-bd8d-6756e488e7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:32:51.750909 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd5041-5c9f-463a-bd8d-6756e488e7fc admin admin] PATH_INFO: `/v3/projects/91eef916bfda4975a8b3d32ad2cc2989` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:32:51.752111 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd5041-5c9f-463a-bd8d-6756e488e7fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=91eef916bfda4975a8b3d32ad2cc2989)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:32:51.754777 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd5041-5c9f-463a-bd8d-6756e488e7fc admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:32:51.847997 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-01bd5041-5c9f-463a-bd8d-6756e488e7fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91eef916bfda4975a8b3d32ad2cc2989', 'request_id': 'req-01bd5041-5c9f-463a-bd8d-6756e488e7fc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:51.854020 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-01bd5041-5c9f-463a-bd8d-6756e488e7fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91eef916bfda4975a8b3d32ad2cc2989', 'request_id': 'req-01bd5041-5c9f-463a-bd8d-6756e488e7fc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:51.916479 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-01bd5041-5c9f-463a-bd8d-6756e488e7fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91eef916bfda4975a8b3d32ad2cc2989', 'request_id': 'req-01bd5041-5c9f-463a-bd8d-6756e488e7fc'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:51.931783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-01bd5041-5c9f-463a-bd8d-6756e488e7fc admin admin] The token cache is being invalidate because project 91eef916bfda4975a8b3d32ad2cc2989 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81479) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:32:51.931783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-01bd5041-5c9f-463a-bd8d-6756e488e7fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:51.931783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-01bd5041-5c9f-463a-bd8d-6756e488e7fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:32:51.931783 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1457/2946] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:32:51 2022] DELETE /identity/v3/projects/91eef916bfda4975a8b3d32ad2cc2989 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:33:01.442865 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=37,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:33:01.924284 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:34:26.909297 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa975b7-0cf1-4a16-b08f-f66f838f5ecd admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:34:26.910822 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5fa975b7-0cf1-4a16-b08f-f66f838f5ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:34:27.016906 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa975b7-0cf1-4a16-b08f-f66f838f5ecd admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:34:27.083059 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53212eb1-6cab-44a4-9469-e8f37883f856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209795285384486183cda455fe316bc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:34:27.083711 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53212eb1-6cab-44a4-9469-e8f37883f856 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:34:27.083944 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53212eb1-6cab-44a4-9469-e8f37883f856 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:34:27.084217 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53212eb1-6cab-44a4-9469-e8f37883f856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:34:27.086009 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-53212eb1-6cab-44a4-9469-e8f37883f856 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:34:27.098965 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53212eb1-6cab-44a4-9469-e8f37883f856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:34:27.100551 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53212eb1-6cab-44a4-9469-e8f37883f856 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:34:27.112120 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1490/2947] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:34:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:34:34.067374 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd5041-5c9f-463a-bd8d-6756e488e7fc admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:34:34.068833 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-01bd5041-5c9f-463a-bd8d-6756e488e7fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:34:34.126083 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd5041-5c9f-463a-bd8d-6756e488e7fc admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:34:34.129600 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445ea4d-0d92-4874-8aa7-b1fe1734ae57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf6798f89acf44e897c481924edd9047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:34:34.130188 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445ea4d-0d92-4874-8aa7-b1fe1734ae57 service neutron] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:34:34.130462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445ea4d-0d92-4874-8aa7-b1fe1734ae57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:34:34.130790 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445ea4d-0d92-4874-8aa7-b1fe1734ae57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:34:34.132547 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a445ea4d-0d92-4874-8aa7-b1fe1734ae57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:34:34.196218 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445ea4d-0d92-4874-8aa7-b1fe1734ae57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:34:34.197655 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445ea4d-0d92-4874-8aa7-b1fe1734ae57 service neutron] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:34:34.207432 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1458/2948] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:34:37.107399 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=71 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:34:37.733048 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53212eb1-6cab-44a4-9469-e8f37883f856 service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:34:37.733996 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-53212eb1-6cab-44a4-9469-e8f37883f856 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:34:37.808918 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53212eb1-6cab-44a4-9469-e8f37883f856 service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:34:37.814752 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9007d-46ad-4d0a-bcbd-0df50b96dc32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6074872303d3468fb566378fa34e8bea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:34:37.815378 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9007d-46ad-4d0a-bcbd-0df50b96dc32 service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:34:37.815650 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9007d-46ad-4d0a-bcbd-0df50b96dc32 service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:34:37.815917 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9007d-46ad-4d0a-bcbd-0df50b96dc32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:34:37.817769 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7db9007d-46ad-4d0a-bcbd-0df50b96dc32 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:34:37.830157 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db9007d-46ad-4d0a-bcbd-0df50b96dc32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:34:37.832742 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db9007d-46ad-4d0a-bcbd-0df50b96dc32 service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:34:37.847854 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1491/2949] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:34:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:34:41.417467 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445ea4d-0d92-4874-8aa7-b1fe1734ae57 service neutron] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:34:41.420845 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-a445ea4d-0d92-4874-8aa7-b1fe1734ae57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:34:41.431094 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445ea4d-0d92-4874-8aa7-b1fe1734ae57 service neutron] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:34:41.436751 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef1ff4-ad8f-4b87-b4e5-4da228b89f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:34:41.437277 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef1ff4-ad8f-4b87-b4e5-4da228b89f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:34:41.437557 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef1ff4-ad8f-4b87-b4e5-4da228b89f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:34:41.437828 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef1ff4-ad8f-4b87-b4e5-4da228b89f16 admin admin] PATH_INFO: `/v3/users/79d29aa9482f4c1f9b1554333d85d80a` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:34:41.439150 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef1ff4-ad8f-4b87-b4e5-4da228b89f16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79d29aa9482f4c1f9b1554333d85d80a)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:34:41.441104 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef1ff4-ad8f-4b87-b4e5-4da228b89f16 admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:34:41.620297 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-15ef1ff4-ad8f-4b87-b4e5-4da228b89f16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79d29aa9482f4c1f9b1554333d85d80a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:41.621030 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-15ef1ff4-ad8f-4b87-b4e5-4da228b89f16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79d29aa9482f4c1f9b1554333d85d80a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:41.633493 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-15ef1ff4-ad8f-4b87-b4e5-4da228b89f16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79d29aa9482f4c1f9b1554333d85d80a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:41.698710 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-15ef1ff4-ad8f-4b87-b4e5-4da228b89f16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79d29aa9482f4c1f9b1554333d85d80a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:41.699355 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-15ef1ff4-ad8f-4b87-b4e5-4da228b89f16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79d29aa9482f4c1f9b1554333d85d80a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:41.712615 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1459/2950] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:34:41 2022] DELETE /identity/v3/users/79d29aa9482f4c1f9b1554333d85d80a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:34:41.961569 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9007d-46ad-4d0a-bcbd-0df50b96dc32 service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:34:41.973894 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9007d-46ad-4d0a-bcbd-0df50b96dc32 service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:34:42.031179 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:34:42.031802 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:34:42.032075 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:34:42.032401 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604 admin admin] PATH_INFO: `/v3/projects/fd1ef1a189d5485fb810e17c945b295f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:34:42.033494 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd1ef1a189d5485fb810e17c945b295f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:34:42.035429 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:34:42.146323 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd1ef1a189d5485fb810e17c945b295f', 'request_id': 'req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:42.152591 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd1ef1a189d5485fb810e17c945b295f', 'request_id': 'req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:42.235764 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd1ef1a189d5485fb810e17c945b295f', 'request_id': 'req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:42.246705 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604 admin admin] The token cache is being invalidate because project fd1ef1a189d5485fb810e17c945b295f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:34:42.247016 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:42.247634 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:42.249885 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1492/2951] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:34:41 2022] DELETE /identity/v3/projects/fd1ef1a189d5485fb810e17c945b295f => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:34:42.253939 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef1ff4-ad8f-4b87-b4e5-4da228b89f16 admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:34:42.255485 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-15ef1ff4-ad8f-4b87-b4e5-4da228b89f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:34:42.269267 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef1ff4-ad8f-4b87-b4e5-4da228b89f16 admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:34:42.286806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:34:42.326806 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e187a59-31fd-4295-8d64-21c526b37a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:34:42.327537 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e187a59-31fd-4295-8d64-21c526b37a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:34:42.327717 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e187a59-31fd-4295-8d64-21c526b37a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:34:42.328018 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e187a59-31fd-4295-8d64-21c526b37a4f admin admin] PATH_INFO: `/v3/users/99738b08189e43879f4ec96ea75fb82a` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:34:42.329155 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e187a59-31fd-4295-8d64-21c526b37a4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=99738b08189e43879f4ec96ea75fb82a)` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:34:42.331097 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e187a59-31fd-4295-8d64-21c526b37a4f admin admin] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:34:42.530890 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3e187a59-31fd-4295-8d64-21c526b37a4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99738b08189e43879f4ec96ea75fb82a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:42.531670 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3e187a59-31fd-4295-8d64-21c526b37a4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99738b08189e43879f4ec96ea75fb82a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:42.543674 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3e187a59-31fd-4295-8d64-21c526b37a4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99738b08189e43879f4ec96ea75fb82a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:42.595824 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3e187a59-31fd-4295-8d64-21c526b37a4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99738b08189e43879f4ec96ea75fb82a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:42.596462 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.notifications [None req-3e187a59-31fd-4295-8d64-21c526b37a4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99738b08189e43879f4ec96ea75fb82a'} {{(pid=81479) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:42.609743 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1460/2952] 10.222.0.30 () {58 vars in 1224 bytes} [Thu Oct 6 08:34:42 2022] DELETE /identity/v3/users/99738b08189e43879f4ec96ea75fb82a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 06 08:34:42.849066 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:34:42.850777 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:34:42.865503 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36e2e8-61c7-4b4d-bdec-1a83f8248604 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:34:42.889690 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:34:42.932593 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3064c69-4532-4d97-aac9-eab38ac12d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93747f497a1f4adcb4b8ddb03adfe01c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f2d30f50f03441f1a79068b287cb95c1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:34:42.933216 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3064c69-4532-4d97-aac9-eab38ac12d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:34:42.933485 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3064c69-4532-4d97-aac9-eab38ac12d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:34:42.933750 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3064c69-4532-4d97-aac9-eab38ac12d2c admin admin] PATH_INFO: `/v3/projects/d75a8b2600804cb593bcabaff5b0a4b7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:34:42.934836 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3064c69-4532-4d97-aac9-eab38ac12d2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d75a8b2600804cb593bcabaff5b0a4b7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:34:42.936894 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3064c69-4532-4d97-aac9-eab38ac12d2c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:34:43.031430 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b3064c69-4532-4d97-aac9-eab38ac12d2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd75a8b2600804cb593bcabaff5b0a4b7', 'request_id': 'req-b3064c69-4532-4d97-aac9-eab38ac12d2c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:43.037417 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b3064c69-4532-4d97-aac9-eab38ac12d2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd75a8b2600804cb593bcabaff5b0a4b7', 'request_id': 'req-b3064c69-4532-4d97-aac9-eab38ac12d2c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:43.110286 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b3064c69-4532-4d97-aac9-eab38ac12d2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd75a8b2600804cb593bcabaff5b0a4b7', 'request_id': 'req-b3064c69-4532-4d97-aac9-eab38ac12d2c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:43.121554 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b3064c69-4532-4d97-aac9-eab38ac12d2c admin admin] The token cache is being invalidate because project d75a8b2600804cb593bcabaff5b0a4b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 06 08:34:43.121869 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b3064c69-4532-4d97-aac9-eab38ac12d2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:43.122540 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b3064c69-4532-4d97-aac9-eab38ac12d2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 06 08:34:43.124866 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1493/2953] 10.222.0.30 () {58 vars in 1233 bytes} [Thu Oct 6 08:34:42 2022] DELETE /identity/v3/projects/d75a8b2600804cb593bcabaff5b0a4b7 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 06 08:34:52.605577 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:34:53.118312 dv-cinder-860212-3-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 06 08:37:51.802318 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e187a59-31fd-4295-8d64-21c526b37a4f admin admin] Authenticating user token {{(pid=81479) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 06 08:37:51.804335 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3e187a59-31fd-4295-8d64-21c526b37a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:37:51.873712 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e187a59-31fd-4295-8d64-21c526b37a4f admin admin] Validating token access rules against request {{(pid=81479) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 06 08:37:51.932499 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c822f-147e-458f-a7fe-ae95111c39ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6074872303d3468fb566378fa34e8bea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '399f0095cc8b4c64893ccc75af13c52d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81479) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 06 08:37:51.933101 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c822f-147e-458f-a7fe-ae95111c39ee service placement] REQUEST_METHOD: `GET` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 06 08:37:51.933381 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c822f-147e-458f-a7fe-ae95111c39ee service placement] SCRIPT_NAME: `/identity` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 06 08:37:51.933682 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c822f-147e-458f-a7fe-ae95111c39ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81479) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 06 08:37:51.935529 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.fernet_utils [None req-3e1c822f-147e-458f-a7fe-ae95111c39ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81479) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 06 08:37:51.946648 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c822f-147e-458f-a7fe-ae95111c39ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 06 08:37:51.948259 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c822f-147e-458f-a7fe-ae95111c39ee service placement] RBAC: Authorization granted {{(pid=81479) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 06 08:37:51.958285 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: [pid: 81479|app: 0|req: 1461/2954] 10.222.0.30 () {60 vars in 1365 bytes} [Thu Oct 6 08:37:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 06 08:38:01.953779 dv-cinder-860212-3-iscsi devstack@keystone.service[81479]: DEBUG dbcounter [-] [81479] Writing DB stats keystone:SELECT=58 {{(pid=81479) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}